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PartI. Survey of the Problems That Must Be Solved in Future Systems 


HUMAN DECISIONS IN MISSILE SYSTEMS 


John E. Mangelsdorf 
Missiles and Space Division, Lockheed Aircraft Corp., Sunnyvale, Calif. 


The development of weapon systems for the free world is an inescapable fact 
of the society in which we live today. The development of these systems in 
the future involves management system problems that must be solved if the 
required technical progress is to continue. It is the purpose of this paper to 
indicate the critical problems in communication between personnel and to sug- 
gest a form that the solution might take. 

An essential distinction between ‘operational systems” and ‘development 
systems” must be made at this point. Operational systems are known by such 
names as Polaris, Thor, and Atlas, and are designed for operation by personnel 
of the armed services. Development systems are known by such names as 

~ Lockheed, Convair, and Douglas, and have as their task the provision of opera- 
tional systems. To emphasize the system characteristics of the communication 
problems, development system is used in preference to the more familiar term 
- management organization. 

The primary source of these communications problems is the sharply in- 
- creasing complexity of weapon systems. As shown in FIGURE 1, the trend in 
weapon systems is an accelerated growth in complexity. This complexity 
stems from the technical sophistication required to achieve planned objectives 
_ of a progressively more ambitious scope. An increase in the complexity of the 
operational system entails increased costs, lead time, and numbers of personnel 
on the part of the development system, that is, the management organization 
_ whose task it is to provide the operational system. FIGURE 2 shows the charac- 
ter of development system costs as they are affected by operational system 
< complexity. Today’s major systems are costly enough, although the United 
States and the Union of Soviet Socialist Republics are still able to finance them. 
Bi: we fail to solve our critical problems, we may well expect that within 10 to 

20 years further complexity will become prohibitively expensive. 
The extent of the communications and, hence, the decision-making problem 
- encountered in a development system even today, is well illustrated by the 
interfaces integral to a representative missile development organization with 
100 departments. There are 4,950 interfaces resulting from the existence of 

100 departments in such an organization. ; 

-  Ficure 3 graphically illustrates this point. If we define a department as a 
group of 10 or more individuals, we find that 100 departments are by no means 
unusual today. These 4,950 interfaces, it must be emphasized, are calculated 
on the basis of organizational units only, not as the number of people within 
each unit. The totality of interfaces actually experienced is a function, there- 
fore, of the number of people in each department with interface responsibilities. 
A significant concept arises in connection with these personnel interfaces 
and, furthermore, provides an understanding applicable to one solution that 
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can be foreseen for the communications problem. Of the thousands of people 
normally found in a development system of this scope, only a small proportion 
will contribute to significant decision making in weapon system design. By 
significant decision making is meant the selection, from several alternatives, of 


SYSTEM COMPLEXITY (OR SOPHIS- 
TICATION) IN UNDEFINED UNITS 
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TIME 
Ficure 1. Growth in the complexity (or sophistication) of weapon systems with time. 
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a course of action having a direct effect upon the progress of development of 
the system. Examples of such decision making are: whether to use solid or 
liquid propulsion; whether to use manual or automatic checkout equipment; 
when to schedule the mth launch; what type of guidance system to employ; and 
what kind and number of people will be required to operate the system. Per- 
sonal experience shows that less than 10 per cent of development system person- 
nel are engaged in this type of significant decision making. 
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Ficure 3. Interface matrix for 100 development system groups. 


The remaining 90 per cent or more of development system personnel operate 
in accordance with detailed instructions, plans, prints, directives, and schedules 
to produce a product meeting well-defined criteria of excellence. Both the 
prototype model and the criteria of excellence are supplied from soe cei 
the “typical” worker in a development system. Personnel in this typical 
category include turret lathe operators, drill press operators, platers, milling 
machine operators, electronic assembly men, typists, and file clerks. A decis- 
ion model for such personnel is offered in FIGURE 4, which shows the develop- 
ment of their product in conformity with the program governing their work. 
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It may be seen, therefore, that essentially no critical decision-making problems 
exist for individuals falling into this category of development system personnel. 

For the personnel manning well-designed operational systems, even fewer 
decision-making problems exist. High reliability of the personnel aspects of an 
operational system requires that essentially all decisions be made in advance 
for these personnel. The decision to employ the operational system is, of 
course, made outside the missile system itself by a chief of state or some other 
duly authorized authority. Once such a system has had its “button pushed, 
a series of events occur in mechanistic fashion in accordance with a plan known 
as the ‘‘countdown.” The performance of personnel varies according to distri- 
bution functions that have their corresponding means, variances, and limits. 
Performance, therefore, is predictable in the statistical sense. 
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FicurE 4. Decision model for most missile system development personnel. 


The typical task of operational personnel is to discriminate between the 
go/no-go alternatives displayed at their work stations, and to undertake pre- 
viously-planned action appropriate to each alternative. In this sense these 
individuals do not decide anything other than what the displayed condition 
portends. Their equipment, instructions, and training provide the basis for 
this interpretive decision and for their consequent action. Indeed, an appar- 
ently high-level decision by the launch director to scrub vehicle No. 12 and sub- 
stitute vehicle No. 14 is, and should be, the result of (1) recognition that vehicle 
No. 12 meets the no-go criteria, and (2) knowledge that the plan specifies sub- 


stitution of vehicle No. 14 when and if No. 12 is scrubbed. FicurE 5 is offered ~ 


as a model for the functioning of operational missile system launch crews. 
These last two decision models were conceived with the help of insights devel- 
oped by L. J. Fogel in his paper in this monograph. 

By accepting the concept of the creative minority we can delimit the com- 
munications and decision-making problem, for we have seen that operational 
system personnel and most development system personnel may be excluded 
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from the ranks of significant decision makers. It is for the minority that im- 
proved communication techniques must be found, as shown by the definition 
of the significance of their decision making. 

The means of communications available today for this creative minority are 
primitive. These personnel communicate by writing on paper, talking face-to- 
face, or speaking over telephones. Except for the lack of telephones, the 
ancient Egyptian had comparable facilities. 
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FicureE 5. Decision model for operational missile system launch.crew. 
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5 ; Ficure 6. Block diagram of information system for development system personnel. 


It is patently impossible to use the spoken word to handle all essential com- 
munications, and paper is therefore the only remaining alternative. However 
itis also impossible for every minority member to send copies of every decision 
~ to every other minority member, or even to every other department head. ; The 
current practice is therefore to exercise “judgment” and to send copies to 
individuals who “need” them or “ought” to have them. In systems with 
nearly five thousand interfaces it is inevitable that such judgment is not infal- 
 lible. vat cof 

_ The result of these communication system constraints 1s that today s minor- 
ity must spend a phenomenal amount of time waiting for information; tracking 
the source of specific information, developing “informal information channels, 
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_ personnel, and improved methods of information display. 
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and coordinating information. Days may pass until a requested document is 
finally obtained, and then it may prove inapplicable to the problem at hand. 
Such examples might be cited at great length; the blame is usually placed on 
the “system,” as indeed it should be. 

Management, for its part, constrained by today’s primitive communication 
tools, has but four means at its disposal to improve development system opera- 
tion. These are (1) to issue directives, (2) to revamp organization charts, (3) 
to reassign personnel, and (4) to add more personnel. The apparently be- 
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Ficure 8. Control panel for information system terminal unit (artist’s concept). 


wildering series of reorganizations, personnel reassignments, new directives, 
and new personnel associated with modern weapon system programs should 
be viewed as the exercise of the only alternatives available to today’s manage- 


~ ment. 


A means must be found, therefore, to relieve the creative minority of an 
increasing burden resulting from the sophistication of new systems. Unless 
such means are found, we shall be reducing the rate of contribution of this group 
even as we ask for increased performance. Solution of this problem for the 


creative minority may be realized by providing easier access to information 


required for significant decision making, faster flow of information between 
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One step that might be taken to effect these changes is the introduction of 
“Information System Equipment,” based on the assumption that all physical 
exchange of paper as a medium of communication can and should be eliminated 


TABLE 1 
Some FUNCTIONAL CAPABILITIES OF INFORMATION SYSTEM TERMINAL UNIT 


Control options ; 
(a) Request specific technical information. _ 
(b) Request specific administrative information. _ ‘ : 
(c) Request delayed playback of message originating from this unit. _ 
(d) Enable access to the display(s) of this unit from other terminal unit(s). 
(e) Request delayed playback of message(s) from other terminal unit(s). 
(f) Record or copy incoming/outgoing messages. 
(g) Enter data into information system. 
(h) Translate voice message to typed copy. ; 
(i) Search selected listings (incoming or outgoing correspondence, specific catalogue sets, 
etc.) 
(j) Select desired document, correspondence. 
(k) Control speed of advance or retrace through document. 
(l) Notify information system that terminal unit will be untended. 
(m) Dial other desired terminal unit(s). 
Display options 
(a) Technical information. 
(6) Administrative information. 
(c) Real-time or delayed playback display of message(s) originating from this unit. 
(d) Real-time or delayed playback display of message(s) from other terminal unit(s). 
(e) Display of information originating from central unit (“tickler file,” estimated ‘“‘best”’ 
time(s) of access to desired other terminal unit(s), etc.). 
(f) Display of catalogues, listings, or other documents in storage in central unit. 


TABLE 2 
EXAMPLES OF ANALYSES OF WHICH CENTRAL UNIT Is CAPABLE 


Delay time analysis for system personnel 
Time to return “telephone” call 
Time to indicate approval 
Time to acknowledge receipt of information 
Time to obtain information 
Time to complete assigned task 
Link analysis 
Who is linked with whom in actual practice 
Link strength 
Link frequency 
Qualitative characteristics of the link 
Sequential dependency characteristics 
Decision-making analysis 
Decision source 
Decision adequacy 


for minority personnel. It is desirable for present purposes to consider none _ 
of today’s restrictions of costs or feasibility. A block diagram of this system 
is shown in FIGURE 6 and consists of twe basic components: an Information 
System Terminal Unit (or simply ‘Terminal Unit”), and an Information Sys- 
tem Central Analysis, Storage, and Monitoring Unit (or simply “Central 
Unit”). The Terminal Unit serves two primary purposes: communication 
between people and information retrieval. The Central Unit serves three 
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primary purposes: analysis of development system information and operations, 
information storage, and system monitoring. 

The Terminal Units are available in Executive and Regular models, and a 
close view of the Regular model (FIGURE 7) shows a display screen and control 
panel. An artist’s conception of some of the elements of the control panel is 
shown in FIGURE 8. Basically, the purpose the Terminal Unit is to allow an 
individual to summon to the screen any system information, either by voice 
~ instruction or by button-pushing and, correspondingly, the insertion of informa- 
tion into the system and/or communication with other members of the system. 
A more complete list of functions the Terminal Unit is capable of implementing 
is Shown in TABLE 1. 

The Central Unit receives inputs from Terminal Unit positions, and these 
inputs are tagged according to date, time, originator, addressee, and subject, 
and put into storage for subsequent retrieval, analysis, and monitoring. Access 
to many of the analyses, as illustrated in TABLE 2, would be limited to higher 
management. Such analyses, based on a quantity and quality of data un- 
available from today’s systems, would assist in implementing the customary 
managerial functions. 

In summary, we have seen that critical decision making problems exist for 
neither operational system personnel nor for the majority of development sys- 
tem personnel. A critical problem, however, does exist for the creative minority 
element in development systems, and relief of the situation by providing easier 
and more rapid communications, perhaps along the lines suggested, is manda- 
tory if future objectives are to be realized. 
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HUMAN DECISIONS IN COMMAND CONTROL CENTERS 


Marvin Adelson 
Ground Systems Group, Hughes Aircraft Company, Fullerton, Calif. 


Introduction 


Men have been designing command control systems for as long as they have 
found it necessary to combine for the accomplishment of almost any mission, 
military or otherwise. As soon as a structure was established for the giving 
and taking of orders and for the transmission of information in a more or less 
systematic way between members of even the most rudimentary organization, 
the essentials of a command control system had emerged. 

As long as the tasks to be performed remained relatively uncomplicated, the 
time available for their completion relatively great, the risks associated with 
failure relatively low, and the size of the group relatively small, the formal 
relations among the elements of the organization in question could remain 
comparatively few and simple. Technological advances have made the mis- 
sions and tasks more complex, but they have concurrently provided the means 
for operating under more complex rules in a more demanding world. While it 
is necessary to recognize the capability provided by modern techniques, it is 
important to avoid being misled by the complexity and magnitude of automated 
and semiautomated systems into regarding the problems as entirely new. 


Basically, they are elaborations of former problems, newly emphasized by the . 


new environment. 

The modern problem of command is distinguished from earlier ones by the 
rate at which information must be handled and decisions made. This distinc- 
tion may be clarified by dividing the problem into its two readily observable 
aspects: (1) complexity and (2) urgency. (While the ensuing discussion de- 
scribes the military case, analogies in the commercial, industrial, and govern- 
mental fields are not hard to find.) 


(1) That a given decision both involves more data and implies more conse- © 


quences than it used to follows from the increased scope of the “battlefield,” 
the greater range and effectiveness of weapons, the greater variety of alterna- 
tive actions available, the larger amount of mobility both possible and desirable, 
the increased interdependence among line, staff, and support activities, and 
improved methods of prediction and planning. 

(2) The increased urgency for proper decisions results from the reduced time 
required for weapon delivery, the magnitude of available destructive effect, the 


irreversibility of commitments, and the possible suddenness of onset of the 
conflict state. 


Command decisions depend (hopefully) upon data either currently supplied | 


or stored. This paper attempts to describe the classes of decision required in 
command, which may be thought of as the process of achieving desired objec- 
tives through selective commitment of available resources. 


_ 


Command centers are typically nodes in networks constituting command 
control systems. The notion of a “center” must not be misinterpreted as" 


implying centralization of decision. It simply indicates a place where some 
726 
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decisions are made. Some centers are simple, consisting of a single decision 
element. Others are complex, containing elaborate networks. 

Definition of system. A system is to be thought of for present purposes as a 
collection of elements organized to perform a set of functions, during a spec- 
ifiable era, in an environment known only uncertainly. “Element” refers to a 
more or less inseparable collection of properties at any of a number of levels of 
discourse. In the present context men and machines are considered elements. 
The characteristics of systems that deserve to be stressed are the organization 
of the elements, the uncertainty under which they characteristically operate, 
and the fact that there is a purpose for which they are intended. In general, 
systems are embedded in larger systems and embed smaller ones. 


Classes of Decision 


A decision will be regarded as a selection from among alternatives. An 
action is a commitment of resources. An action decision is therefore a selection 
_ from among a set of alternative resource commitments. The resources avail- 
_ able for commitment by a commander include men, money, material (including 

weapons), energy, information, and time. In the following it is not to be in- 

_ ferred that decisions are necessarily made in the order described. On the 
_ contrary, the most characteristic operation of command centers involves con- 
- tinual and concurrent decisions of all kinds. 

_ State of the world. Under the assumption that a good decision is an informed 
decision, it is at least desirable for the decider to be aware of the present state 
- of the world. This state is described by the values of a set of relevant variables 
that include not only absolute values but indications of departure from ex- 
_ pected, desired, or required values. The question of how these expectations are 
_ built up is an extremely interesting one that is, of course, receiving considerable 
scrutiny. Since a complete description of the world can be neither achieved 
“nor used, the decider must operate upon information in terms of classes that 
_ permit orderly storage and utilization. The selection of meaningful classifica- 
tions is one of the challenging decision problems of command systems. 
Inasmuch as the decision process is extended in time, the decider must sense 
~ the change in relevant variables over some period of the recent past, the length 
_ of which depends on each particular piece of information involved. This is 
_ important for two reasons. First, any given change may be a trigger to action. 
Second, for sampling purposes the decider is forced to estimate what may be 
called the relative stationarity of the process being observed. For if he is to 
~ generate estimates based upon statistics he must determine an appropriate 
~ sampling interval or sample size. 
Since estimates of the present state are used to contribute to action selection 
and since initiation of action generally takes time, it is important for the decider 
_ to predict changes that are likely in the future. Of course, unpredictability 
_ grows as a function of time. Moreover, the future state is partly a function 
_ of the action vectors chosen by the decider, his antagonist, or others. ; 
The military decider needs information on actually or potentially hostile 
agents’ capabilities, limitations, disposition of resources, policy, and other 
“special characteristics. He must take account of the relevant neutral environ- 


ark 
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ment (physical, economic, or other), and he must be aware of his own strength 
and limitations, disposition of his resources, policy, and the special character- 
istics of his subordinate commanders and his staff. He needs to know the 
present state internal to the center, at other centers, and in other systems with — 
which his system interacts. The problems of acquiring information of these 
three kinds are quite different, since the sources are polarized negatively, 
neutrally, and positively. 

Since the decider’s knowledge of the world is always imperfect, his descrip- 
tions must be partly incomplete, partly statistical in nature. Thus he is inter- 
ested in knowing-en which variables he has information and on which variables 
he is lacking information, and he must estimate the probability distributions 
associated with all stochastic variables relevant to his problem. It is in 
connection with this last kind of knowledge that information on capabilities, 
limitations, interest, preferences, and experiences of his opponents is valuable, 
for they shift his estimates of probabilities. 

Available alternatives. At any given time the decider must analyze what 
actions are available to him. The set of availableactions is a function of time. 
Some changes may be quite sudden, due to such factors as enemy action, 
political activities, and technological breakthroughs; or they may be gradual, 
due to obsolescence, price changes, and the like. In any real-time situation, 
the various aspects of complex action unfold at different rates, so that the 
consequent constraining effects vary progressively. Moreover, the actions 
available at a given time clearly depend upon the decisions made up to that 
time. In this situation, the commander must decide what new alternatives 
have been introduced and what old ones eliminated. It is important to note 
that obtaining additional information about any or all of the categories of 
interest is one kind of action, and that the alternatives effectively available 
depend upon the state of knowledge of the decider. 

The decider is interested not only in his own alternatives but in those of the 
competitors, opponents, or enemies with whose alternative actions he will have 
to cope and which he, therefore, must predict; he also needs information on the 
decisions of others who may affect the processes, either as catalysts or as deter- 
miners of boundary conditions or else by becoming participants. Since in- 
formation is expensive, the measures required for its acquisition deprive a! 
commander of degrees of freedom just as effectively as any other resource 
commitment. One of the problems in training deciders is in convincing them 
that information is a commodity for which it is worthwhile to sacrifice a certain 
amount of their operational capability. 

Predicted outcomes. The possible outcomes of a selected action diverge in 
time, as do the branches of a tree in space. Even during a limited interval 
the possible consequences of an action may be too numerous to list, and they 
may continue to unfold indefinitely and to interact with other factors. Thus, 
as a basis for selecting actions, the predicted outcome is a source of considerable 
uncertainty. Nevertheless, without a modicum of predictability, purposive 
behavior is impossible. 

. The relevant aspects of the consequences of an action include the expected 
value” as a function of time; the new state of the world, expressible stochas- 
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tically; and a new set of available alternatives (which may be subsumed under 
the description of the world but is better distinguished for present purposes). 

The outcomes of decisions are functions of a set of relevant variables of 
which the identities or distributions may be partially unknown. At the present 
state of the art of war, management, or government we do not know all the 
relevant variables, nor are we able characteristically to specify, for those we 
do know, the distribution functions as a function of time. To gain some in- 
sight into the possible outcomes of complicated processes similar to those we 
expect to encounter in practice, such activities as war games, or business games, 
have become fashionable as part of the process of planning. In using these 
games, explicit or implicit decisions are made about the applicability of the 
outcomes of the games as estimators of the outcomes of actual operations, and 
about the degree to which the game simulates the operational situation. 

Objectives and criteria. The operation of command control systems is pur- 

poseful; that is, it has objectives. This implies that a set of decision criteria 
are being applied, whether or not they can be identified, but this set is not 
_ necessarily the same at all times for a given system. 
Here again the military commander may be taken as an interesting example. 
- After a particular kind of attack his criterion or ordering process on the set of 
available action alternatives may be very different from that he held before 
the attack. Moreover, it is not at all clear that this criterion could have been 
anticipated in all cases. It will depend, among other things, upon his estimate 
of the desires, criteria, or ordering processes of others besides himself, such as 
his commander and his commander’s commander. 

Moreover, and this point is interesting to those who feel that the place of 
human beings in future systems is tenuous at best, objectives and criteria are 
not necessarily computable functions. If there can be said to be a field for 
technological improvement of human beings in connection with command 


control centers, the direction for such improvement seems to be in what is 


wanted, how it is wanted, and how the want is formulated, so that it can be 
realized by the assemblage of men and machines that constitute the system, 


Conclusions 


The foregoing discussion gives only an incomplete idea of the complexity of 


the decision problems faced at the nodes in a command control system. Since 


‘ -information-gathering and information-keeping resources must always be 
» limited, it is not at all evident that at any given time all relevant information 


aa about the state of the world is or can be made available. We need to know 


what constitutes an “acceptable” set of information. Nor is it obvious that a 


“best”? decision can be associated in advance with each discriminable state of 


the world, so that when the one occurs the other can be automatically sum- 
-moned. It is not even clear that a satisfactory description can be given of the 
likely outcomes of each of the available alternatives at any instant. Even the 
_ objectives, on the basis of which preferred subsets of available alternatives 
‘could be and are formed, are subject to progressive development, not neces- 
sarily describable in advance. Moreover, these four kinds of decisions interact 


vigorously. 
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The wonder is that decisions better than chance seem to occur. The paradox 
is that I seem to think I can tell what is better than chance. 

However, decisions are made. Objectives can be stated, and they do re- 
main valid for a while. The state of the world can be assessed, and the as- 
sessment used selectively upon what is at least an implicit set of alternatives. — 
Resources are committed, and organizational changes are made. These things 
are possible because rates of change in the processes to which we are required 
to respond tend to be bounded. The military problem has always been to 
deal with events, actual or possible, in which these bounds are exceeded. The 
hunger for both information and the means to process it effectively and rapidly 
has grown acute in proportion to the magnitude of the change that may be 
accomplished abruptly. 

By virtue of present methods of swift delivery of high yield weapons, we 
may be approaching the limit of increase of rate of change. The command 
control problem may, in certain cases, have become insuperable. In the face 
of bilateral inability to insure adequate decisions, potential antagonists may 
eventually abandon cataclysmic conflict for subtle conflict, the hot for the 
cool war. In subtle conflict, the objective will be to obscure rates of change of 
relevant cues about one’s own actions while detecting and countering those of 
one’s antagonist. To protect these indicators they should be kept below a 
certain attention threshold. This threshold can be kept high by creating a 
series of incidents specifically designed to be obvious and, at the same time, 
incoherent with respect to acts that are instrumental in terms of planned objec- 
tives. While required response speeds may then decrease, the information re- 
quirements for decision will increase to a point where it is unlikely that any net 
decrease in processing rate will be possible. In fact, as our sophistication in the 
use of information increases, it is likely that required rates will continue their 
upward climb. 

It is interesting to note that, should the trend from hot to cool war materi- 
alize as suggested, the military command control problem will grow to resemble 
more and more closely the corresponding business, industrial, and governmen- 
tal problems. 

This paper discusses a problem domain in a way that is intended to integrate 
a number of problems too often seen only in isolation. In choosing among the, 
levels of discourse at which it is possible to formulate such a problem, the cri- | 
terion was to bring the course of the discussion tangent at some point to the 
body of interests of participants in system operation, who make the decisions; 
system analysts who must identify, classify, and assign the decisions; system 
designers who must implement the means for forming the decisions; and theo- 
rists and experimenters whose aim is scientific understanding. The terminol- 
ogy therefore may be somewhat mixed, but that is an acceptable price to pay 
for the attention of those workers in the diverse fields who have substantial 


contributions to make to the solution of the problems of human decisions in 
complex systems. 


Discussion 


_Joun LyMan (Biotechnology Laboratory, Department of Engineering, Univer- 
sity of California at Los Angeles, Los Angeles, Calif.): It appears that the con- 
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tent of this first group of papers has raised three classes of decision problems, as 
follows: (1) problems associated with selecting or “gating” information from 
the environment; (2) problems of collating information with that in human 
storage and establishing criteria for when the information is complete enough 
for a decision to be made; and (3) problems of organization and control of ac- 
tion elements to carry out the effective utilization of available information. 

A further set of problems seems to be evident but was touched on primarily 
by implication rather than directly. This set includes the problems associated 
with selecting goals and setting values for the functions of the systems them- 
selves. Adelson’s paper was particularly relevant to this. It is becoming 
more and more manifest in our time that much that will influence human “ways 
of life” favorably depends on what humans decide to do with their environment 
to modify it in various dimensions. Decisions about such matters depend on 
processes about which little appears to be understood. Accordingly, this ap- 
pears to present a particularly challenging area for research exploration. 

Human observers are known to perceive situations on the basis of past ex- 
perience and instructions for observing. Following this argument, it is specu- 
lated that radical effects on decision outcomes may be shown to be related to 
perception along a ‘‘value” or personal criteria continuum for both simple and 
complex tasks. It is suggested that any complete theory of human decision 
processes must take account of the parameters associated with the formation 
of human judgment criteria, as well as the so-called ‘‘objective” facts. 


Part Il. Theoretical Approaches to the Decision-Making Problem 


LEVELS OF INTELLIGENCE IN DECISION MAKING 


Lawrence J. Fogel* 


Reliability Analysis Group, Convair (San Diego) Division, General Dynamics 
Corporation, San Diego, Calif. 


Introduction 


The advent of complex man-machine systems has focused scientific atten- 
tion on the general properties of automata. Determination of realizability, 
channel capacity and other essential limiting factors have been accompanied 
by the development of rather sophisticated techniques for the analysis and 
synthesis of machinet systems. There has been, however, a distinct lag in 
comparable technology relating to the designed use of the human operator to 
work as part of a man-machine team. 

This deficit becomes even more apparent when the operator’s assigned task 
is more than simple tracking or other conditioned reflex decision behavior. 
Automation, the replacing of lower level human functioning by that of ma- 
chines, is becoming limited by the lack of understanding of what the human 
does in arriving at a decision. Today the designer includes human operation 
in the system only when his knowledge of the problems that the system will 
encounter is incomplete. He makes up for this unknown, and the unexpected, 
by specifying ‘‘a portion of himself” as part of the design. The only true 
justification for including man in modern equipment systems lies in his unique 
value as a decision maker. The greater the degree of automation, the greater 
the burden of responsibility that rests upon the remaining human decision 
making. 

It is considered worthwhile to attempt to quantify the decision-making 
capability in order to characterize human performance better and, as such, 
offer a scale useful in the design of future automated systems. Complex system 
design remains an art until a set of measurements becomes available that will 
describe equally well the performance of both men and machines. Indeed, 
such measurements would also prove of significant value in the design of purely 


mechanized systems or subsystems to meet given logical requirements most 


efficiently. 

The external behavior of any automaton organism (man and/or machine) 
can be fully described in terms of its input stimulus and output response, 
given a particular history of previous experience and behavior.! Such trans- 
duction may be viewed as being a result of internal decision making. The 


purpose of this paper is to offer a quantitative measure for the intelligence of - 


any decision-making transduction. Although the terms intelligence and de- 
cision making will be generalized far beyond their subjective connotation, it is 


—_ 


* Presently on leave to the Office of the Director, National Science Foundation, Wash-— 


ington, D. C. 
+ The term machine is taken to indicate an 


y artificial mechanization, including electronic 
and other types of systems. 
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desirable that they agree with the more usual intuitive notions. The intended 
measurement must prove to be of far greater value than the intelligence quo- 
tient (IQ)* in describing men, machines, or man-machine combinations. 

The intelligence of specific decision making is a property of the particular 
transduction and, as such, it is bounded, but not determined, by the physical 
nature of the automaton and the system in which it is enclosed. That is to 
Say, an automaton-organism need not operate at its highest level of intelligence 
and, in fact, might reach this level only on rare occasions. The amount of 
intelligence that is brought to bear is dependent upon the nature of the system 
within which the automaton finds itself. If only the simplest decisions con- 
tinually satisfy the environment it would be unwarranted (and possibly even 
unwise) to exercise greater intellect. In general, then, the level of intelligence 
is directly related to the decision making that is accomplished. 

‘Before considering an absolute measure for intelligence, it appears necessary 
to determine an ordinal scale of the logical complexity of decision-making be- 
havior. 


Deductive Decisions 


Deductive decisions can be carried out with complete certainty. Deduction 
is accomplished when a particular case is drawn from the general rule as a 
result of some inquiry that falls within the class covered by the rule. To give 
the classical example: Socrates is a man; all men are mortal; therefore, Socrates 
is mortal. In more modern terms: given an input, «;, and an appropriate 
operator, F, the result y; is determined by y; = F(x). 

At the lowest level of logical complexity is the special deductive case of ‘‘no 
output-response”’ transduction. The automaton receives an input stimulus 
but continually remains “silent.” It is important to note that the decision 
belongs only to this ‘‘mute”’ class as long as the lack of output is not a coded 
response in some manner. In general a deductive decision maker is prepro- 
grammed to carry out its operation in a deterministic manner, translating 
each of the possible inputs into some particular related output. The automa- 


~ ton attempts to do only what it has been programmed to do. No matter how 
complex the actual transduction may be—whether it be multivariate, non- 


linear, or the result of operation upon the memory—it is possible, at least 
theoretically, to determine what the output should be from knowledge of a 


_ blueprint of the machine. All variability in the output is due solely to noise 


"arising at any point in the transduction. 


To illustrate, we may cite the simple mechanical control system of a small 
aircraft that receives input signals from the human operator and transduces 
them into control-surface movements. Each control-stick input should always 
result in the same aileron- and rudder-position output. Any gross variation 


in this transduction is the result of noise and yields degraded performance. 


The resulting change of flight path of the aircraft is, in a similar manner, 
the outcome of a purely deductive decision by the aircraft itself, the composite 
i j ith respect 

*TIQi ti sychometric measurement of the performance of a subject wit esp 
toa Bon cst nde “Jaboratory” conditions, the average IQ being arbitrarily taken to 


be 100. 
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of both the control surface input signal and the inertia memory of previous 
control actions and flight. The aircraft response would be exactly predictable 
in the absence of noise and here, again, the only result of imposed environ- 
mental or internally generated noise is uncertainty in the outcome that de- 
grades performance. i 

It is important to realize that deductive decision makers need not be passive. 
In fact the human operator depends to a surprisingly great extent upon con- 
ditioned-reflex decisions that occur below the level of conscious mediation. 
Acquisition of such conditioning is certainly worthwhile in order to achieve 
rapid and dependable response to certain important stimuli (for example, 
emergency signals). Their utility, however, in no way affects their logical 
complexity. Such simple input-output decisions might best be called “mo- 
ronic.” Obviously any logically simple system of automata composed of 
members that make decisions only at the moronic level of logical complexity 
may also be classed as moronic. Regardless of physical size or structural com- 
plexity, the preprogrammed input-output transducer remains the maker of 
moronic decisions, as represented in FIGURE 1. 

At a higher level of logical complexity there are deductive decisions that 
intend to optimize performance with respect to some given set of criteria. The 


FicurE 1. Moronic deductive decision. 


automata accepts information from two separate sources—the reference signal 
and the sensed environment—and combines these under some given rule (set 
of criteria) to yield the particular “best” output response. The information 
derived from the environment is related to previous values of the output con- 
trol signal and thus provides a closed loop through the memory of the environ- 
ment.* The optimizing decision maker compares the two input signals in some 
manner, thus forming a model of the transduction that must yet be accom- 
plished in order to maximize the value of the given measurement as defined 
by the set of criteria. The decision maker is structured to be capable of ac- 
complishing only a certain set of control signal responses. It selects one of 
these as being most appropriate, based on the particular disparity between the 
intended environment and the actual environment as viewed through the cri- 
teria. 

In general the optimizing decision maker is composed of a model and some 
implementing transducer, configured in either of two possible ways. As shown 
in FIGURE 2, the optimization is accomplished by designing the model of the 
desired transduction so that a properly revised reference signal is generated 
and furnished to the implementing moronic transducer. The usual closed-loop 
servo system falls within this class of optimizing decision maker. The error 
is formed by taking the difference between the reference variable and the 


* The real world memory may be simply in the form of inertia or structural integrity. 


a 
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status of the controlled variable. This error becomes the input to the moronic 
implementing transducer. Such automata continuously attempt to minimize 
the error under the tacit assumption that this will also maximize the value of 
total transduction. 

In an alternate configuration, shown in FIGURE 3, the model is used to deter- 
mine an internal signal that will modify the implementing moronic transduc- 
tion so that its operation on the actual reference signal will yield directly the 
desired output response. This configuration may be called an adaptive opti- 
mizing decision maker. In recent years there has been considerable interest 
in the design and use of such adaptive servo systems. 

Various combinations of these two types of optimizing systems can yield 
somewhat more complex automata that use the model not only to modify the 
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FIGURE 2. Optimizing deductive decision. 
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Ficurr 3, Adaptive optimizing deductive decision. 


ay 


input signal but also to modify simultaneously the transduction of the imple- 
- menting output device. Such design becomes particularly appropriate when 
_ limiting constraints are placed upon the signal range as well as the variability 
of implementing transduction behavior. It is not uncommon to come upon 
multilevel closed-loop systems designed so that a hierarchy of optimizing de- 
cisions enclose one another. Furthermore, complex mechanizations may 
_ encompass a number of importance-weighted parameters and maximize the re- 
sulting figure-of-merit in order to determine an optimal corrective action. Re- 
- gardless of the complexity of the physical configuration there results a system 
_ operating at the same level of logical complexity: that of deductive optimiza- 
tion. In general, any optimizing decision may be viewed as composed of 
_ separate model-determining and moronic-implementing subdecisions. 
_ Deviation of the output-solution from that determined by the program of a 
deductive decision-maker is the effect of noise. Usually this results in de- 
graded performance; however, it is possible to offer specific instances in-which 
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the introduction of noise may aid decision making. To illustrate, consider an 
optimizing decision maker operating within the noiseless domain of pure num- 
bers. The imposed reference criterion is that the output solution be a rational 
number lying between 6.5 and 6.6. The automaton 1s programmed to use the 
four simple arithmetic operations on any of the whole numbers ranging from 
zero to nine as “sensed” from the environment. Successive combinations are 
accomplished through a programmed orderly procedure. In each case the 
related error is determined from the difference between the created number 
and the desired output. 

The automaton initially examines each of the whole numbers and finds that 

a satisfactory solution is not yet available. It therefore proceeds to generate 
combinations forming a new set of possible solutions, these being 1 + 2, 2 + 3, 
3 + 4, and upward. These also fail the test; therefore the other arithmetic 
operators are brought to bear in order, first on digrams, then on trigrams, and 
in ascending succession. The set of possible solutions generated is infinite in 
extent but includes only the denumerable set of rational numbers.* Even- 
tually, a valid solution will be realized, regardless of the specific procedural 
order of programmed investigation. 

It is easy to find a simple criterion that would prevent a valid solution from 
ever being realized, even within this large but specialized domain of generated 
alternative solutions. For example, suppose that the requirement had been 
that the solution be an irrational number lying between 6.5 and 6.6. It is 
obvious that no such number will exist within the generated set of possible 
solutions. At this point it is worthwhile to consider the addition of analog 
noise in the construction of the set of possible alternative solutions. Each 
time a number is taken for combination it is slightly different in value and, 
in fact, some variability is also characteristic of the combining arithmetic 
operators. As a result, each generated combination may be slightly different 
from the noiseless result and from any previous attempts to accomplish the 
same combination. Notice that the addition of noise has released the de- 
numerable set of possibilities into a domain of nondenumerably infinite extent. 
The same deductive optimizing review of the possible alternatives can now be 
accomplished with success, since the entire set of irrational numbers will even- 
tually be generated. : 

It is interesting to note that for any such decision maker there is some op-_ 
timal signal-to-noise ratio in order to attain a valid solution within a minimum - 
time of operation. If the noise level is too great with respect to the signal, a 
great diversity of alternatives will be rapidly generated, covering a wide do- 
main far afield from the original given alternatives. Such “radical” adduction 
may furnish a large number of solutions that require the expenditure of con- | 
siderable time and effort in examination prior to finding a successful solution. | 
In fact it may be that such introduced diversity may skip over the desired 
solution, thereby-greatly increasing the total cost of its final determination. 
On the other hand, too low a noise level will only modify the sensed data in a 
most conservative manner, thus providing a slowly increasing domain of possi- 
ble solutions that is more carefully surveyed as the deductive process continued 


* The solution might first appear in the form (8 X 9/6 + 5). 
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in its review. Obviously there is some “‘best”’ level of noise that will generate 
the required solution at a minimal cost of time and effort. Noise is multi- 
dimensional. Each of the parameters within which a decision maker operates 
may be noisy so that, regardless of the provided constraints, it always remains 
possible to find a solution to a given problem through the useful use of noise, 
provided the given problem is itself consistent and possible of solution in the 
real world. 

At the lower levels of structural complexity noise is almost always a degrad- 
ing factor. It has been shown, however, that at a higher level of logical com- 
plexity noise can serve a useful purpose by releasing the domain of those al- 
ternatives that are available for consideration. The automata may be said 
to have “created” or “invented” new solutions that may then be recognized 
as valuable by the ensuing deductive process. By analogy it might follow 
that the process of human invention is the result of noise internal to brain 
functioning coupled with careful deductive examination to determine which 
of the generated products are of immediate utility (that is, which generated 


_ products satisfy the constraints of “necessity’’). 


The real world is always noisy. The likelihood that this noise will signifi- 
cantly increase the utility of a mute decision is nil. The probability that the 
result of a noisy moronic decision will happen to fall within the desired range 


- of parametric values is somewhat greater. The search for a valid decision by 
~ an optimizing automaton allows even greater opportunity for the useful use 
_ of noise, since the feedback provides a deterministic influence toward reaching 
the desired solution. It appears reasonable to suggest that the greater the 
- logical complexity, the greater the possibility for the use of noise to facilitate 


reaching a successful decision. 


Abductive Decisions 


Abductive decisions usually have some essential uncertainty and yet this 
type of decision is common to everyday experience. Abduction is accom- 
plished when a particular ‘‘causal” factor is to be determined as a result of 
some given outcome “‘effect’’ and a governing rule. In the classical idiom: 
Socrates is mortal; all men are mortal; therefore Socrates isa man. Perhaps, 


‘but Socrates might also bea dog. That is to say, given the resultant, Yi» and 
the operator, F, then the causal factor, ; , is presumed to be determined by 


“%; = F-(y;). Even though uncertainty prevails the decision maker may 


have no better course of action than to use the given rule directly in an in- 
“verse manner. Possibly the greatest danger lies in the decision maker’s not 


“realizing” the degree of uncertainty that is inherent in his abductive decision. 
‘Evidence is presented. The jury is presumed to be sufficiently mature and 
experienced to know patterns of human behavior. Based upon these “rules” 
and the provided evidence, these “good men and true” must abduce the most 
probable causal factor and declare innocence or guilt accordingly. Circum- 
stantial evidence has certainly been known to lead to false conclusions. 
The radar operator sees a particular pip on his scope. His task is to deter- 


“mine if this pip represents a real target or is just the result of noise. ‘The rule 
is provided in terms of his experience using similar equipment systems under 
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similar circumstances. The operator makes an abductive decision, and his 
decision remains dependable only to the extent that his “rule” is useful to dis- 
criminate signal from noise and also covers the actual real world situation en- 
countered. His decision task may be viewed as one of pattern recognition. 
Neglect of the noise level can easily demolish the value of decision making. 

Taking a measurement is an abductive decision. A thermometer provides 
direct evidence only in terms of the amount of expansion of, for example, 
mercury; the implicit rule is the particular structure of the tube and bulb. 
The decision maker uses the indicated scale value and the “‘given” rule de- 
fined by the explicit scale on the tube in order to abduce the corresponding 
temperature. Even the simple measurement of length is indirect, as it is in 
terms of the number of arbitrary units that cover the distance. An abductive 
process is needed to reach the desired conclusion. It is easy to find a wide 
diversity of measurements that, by being indirect, require an abductive de- 
cision because pattern recognition is only one form of measurement. 

Inanimate mechanisms may be programmed to accomplish abductive de- | 
cisions, as demonstrated by a number of recent experiments wherein a digital 
computer constructed logical proofs for stated theorems given the rules of 
manipulation and previously proved theorems.? The adjoint method for in- 
vestigating a control system is also an appropriate example of abductive logic.* 
The adjoint of a time-varying system is that system whose impulse response, 
considered as a function of measurement, is equal to the impulse response of 
the original system when taken as a function of the time at which the impulse 
is introduced. A well-defined procedure exists that allows description of the 
adjoint of a system. The result can then be explored, through the use of an . 
impulse function, to reveal its transfer characteristics and the probable causal 
factors for any known response. 

Such abduction may be viewed as an array of deductive subdecisions, as 
shown in FIGURE 4. The general rule is translated into its inverse, assuming 
this to be possible; the result is used to program the moronic deduction that 
transforms the data at hand into some possible causal factor. This latter | 
subdecision might be viewed as a constructed model of the real world process 
under investigation. There is an interesting similarity between the logical 
configuration shown in FIGURES 2 and 4, each appearing as a reflection of the, 
other. ; 

It is common to require optimal abduction, that is, the determination of the 
most probable cause for a given effect. This can be provided by following 
the above-described abduction with an optimizing deduction to select one — 
that corresponds to the greatest value from among the set of generated possible 
causal factors. The diversity of this set might relate to an unknown constant 
of integration, in the event that the output of the real world process is de- | 
scribed in terms of a convolution integral using the given kernel. 

Abduction need not relate to time-ordered factors linked by causality. In 
the sequential case, however, it is usually found that the greater the span be- 
tween the reference and observed time, the greater the abductive uncertainty. 
Stated in another way, the point of minimum entropy in the perceived world 


corresponds to the present. Entropy increases as the “view” extends farther 
into the past or the future. 
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Another subdecision in the abductive process may relate to the required 
minimum level of uncertainty of the solution.® If there exists such a con- 
straint, it may prove appropriate for the automata to review the available 
data until a sufficient amount accrues to satisfy the requirement for valid 
abduction. Such an uncertainty estimate is most likely to be deductive based 
upon a rule derived from past experience. 

A very large number of abductive decisions fall in the class of statistical 
decision problems. The automata-organism is presented evidence on some 
available sample and upon this basis must determine the nature of the most 
probable parent population in terms of some choice of hypothesis.* The 
literature on this subject is extensive.*7 The field of signal detection theory 

_ has developed upon this foundation. A rather extensive normative theory 
_ has been favorably related to empirical evidence regarding human detection 
_ of signal in noise.’:9!9 Game theory and dynamic programming cover an 
- abductive domain wherein the decision proposes a best course of action (causal 
_ factor) under a given set of cost criteria and a desired ultimate condition.” 
_ Detailed discussion of these topics is considered beyond the scope of this paper. 


MORONIC 
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Ficure 4. Abductive decision. 
"The use of heuristics in abductive decision making can prove to be of dis- 
tinct value. The “reasonable” choice of program can produce a set of possi- 
ble solutions that will more quickly satisfy the imposed constraints. The 
~ heuristic decision maker may benefit from past failure as well as from past 
~ success. The decision may have proved to be negative, and yet it might be 
- possible to extract some elements of positive value in order to improve future 
~ decision making. Heuristics relate to the program of decision and, as such, 
_ may be considered as an overview “rule” derived from the memory of past 
: experience.” ‘The application of the heuristic rule compounds the given rule 
; and usually proves to be of benefit (although it is quite easy to construct 
mathematical problems where intuition fails). Decisions that determine the 
‘nature of the rule under which abductive decision is reached are known as 


‘inductive decisions. 


Inductive Decisions 


Inductive decisions always remain uncertain to some degree. Induction is 
© accomplished when evidence of at least two specific data points is available 
and there is a desire to generalize upon this basis to some governing relation- 
ship. It is impossible to find a method of constructing exact generalizations 


isi i i i rue the decision 
_ * As Jong as the decision considers which of the available hypotheses are true t e 
‘is Ra anehive. On the other hand, the generation of hypotheses is an inductive decision. 
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which will always remain valid. Furthermore, the apparent validity of one 
particular logical pattern in no way offers proof that there may not be some 
other equally valid and perhaps even more appropriate pattern. To quote 


the classical idiom again: Socrates is a man; Socrates is mortal; therefore all — 


men are mortal. Reference to Venn diagrams makes the uncertainty condi- 
tion immediately apparent. In more modern terms: given x; and 4; , find the 
F that might have caused this occurrence or relates the available evidence. 

Probably the fundamental problem of inductive decision making relates to 
the establishment of categories. The two given pieces of evidence are alike, 
but in what manner? The young child views an array of coins. He separates 
those of greater “value” from the rest by choosing the shiny ones. His de- 
cision indicates an implicit generalization “Shiny coins are ‘better’ than dull 
ones.” Without any given categories the inductive organism must generate 
its own set of classes by whatever discrimination means are available. Mac- 
Kay! has suggested how classes may be generated at increasing levels of ab- 
straction through deductive logic operating upon the ability to discriminate 
some fundamental property. The ability to erect a classification at even the 
lowest level may require that the automaton possess a model and choose to 
accomplish a comparison subdecision at least at the nominal level. 

Let it be presumed that an elementary automaton-organism can distinguish 
only the existence of light (versus dark) in its environment. Memory of the 
immediately past state provides access to “has the environment changed?” 
Correlation of such a perceived light-intensity change with exerted effort 
might constitute the basis for the discrimination of movement (two classes: 
moving versus not moving). Movement is a vector and may be classed with 
respect to direction and, through the use of memory, with respect to patterns 
of direction. Patterns of patterns lead to higher and higher levels of abstrac- 
tion. 

In view of the stated requirement for at least a minimum of two data points 
it is interesting to consider the original induction of a newly formed automaton- 
organism that is as yet memoryless. Let it be assumed that no outside in- 
struction or program of classes has been provided. Is it possible for such a 
system to accomplish inductive decision? An affirmative view can be offered. 
All signals that can be distinguished from noise have some internal redundancy.’ 
Only through such redundancy is detection possible (abduction). Accordingly 
even the simplest signal can be divided into at least two data points, thus per- 
mitting some generalization (possibly regarding its source). 


The elementary subdecisions for induction may be viewed as shown in FIG-’ | 


URE 5. Two class-model deductive transducers receive the available informa- 


tion from either the sensed real world, the memory, or both. Their outputs — 
represent only the class characteristic of the input signals. These are then ™ 


compared to determine equality within the range of discrimination of the class 
models. The result of the inductive decision is an hypothesis with respect to 
the entire population from which these signals «; and yi were originally drawn. 
At higher levels of abstraction the class-model transducers become multidi- 
mensional, and comparison can be accomplished in a number of respects at 
the same observation. The result can reach any desired complexity of “hy- 
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pothesis” or “governing rule.” Of course the greater the specificity in each 
dimension and the greater the number of dimensions viewed, the more likely 
it is that the inductive solution will truly represent the real world. 

A more general set of conditions that make possible the existence of a classi- 
fication system have been stated by Uttley:!6 
“1. Each input channel must be always in one of two states—activity and 
inactivity. 

“2, The inputs must be combined in as many ways as possible—ideally in 
all possible ways. 

“3. There must be a unit for every combination of inputs, which indicates 
that every input of the combination is active. A combination of a set of in- 
puts is said to define a pattern of activity. The connections between inputs 

- and units are called counting connections. 

_ “4, Temporal patterns are to be distinct. Each input must pass through a 
_ period of delay; the output of each delay must provide a separate input to 
_ the system of indicating units.” 


4 


CLASS MODEL 


COMPARATOR 


Yi CLASS MODEL 


FicurE 5. Inductive decision. 


At present there is significant interest in heuristic programming. The term 
“heuristic” is taken to mean the accomplishment of “reasonable” decision- 

- making behavior. At the lowest level it is possible to program adaptive be- 
~ havior in such a way that the mechanism will continue to solve the given 
problem by a successively more and more efficient technique (the transduction 
attempts to attain greater value with respect to the given set of over-all cri- 
teria). Such a device is usually not dignified by the adjective “‘heuristic,” 
Ethis being reserved for automaton organisms that appear to accomplish the 
‘same end, but lack the explicit framework of an over-all criteria that justifies 


the modification of program. Note that whether the reprogramming is the 


required “to satisfy the real world environment.” 
___ In living organisms decisions are rarely, if ever, reached on the basis of a 
singular parameter. In the inductive case it might be observed that a multi- 


_ plicity of decision levels are apparent for even the simplest case. Everything 


a 


of concern in developing a program, a rule, or a generalization is related to an 
entire hierarchy of subgoals of various utility weightings. It is suggested 
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that the heuristic human capability is a result of the concurrent generalization 
of the given evidence at various levels resulting in a set of “rules.” If the 
decision reached on the basis of the lowest-level rule proves to be improper, 
there is no need to throw away the generalizations at all higher levels. These 


can be retained to aid in formulation of a new decision based upon the same or 


additional evidence (including the result that the previous solution proved to 
be of little or no value). 

To illustrate with an oversimplified example: given the sequence of numbers 
1, 2, 3, find the governing rule needed to generate the next digit. The de- 
cision maker generates the obvious 4; that is, the rule is taken to be the set 
of whole numbers, 1, 2, 3,4- +. The examiner (representing the real world) 
provides a feedback that indicates this is not correct. The decision maker 
approaches the same data once again. No consideration is given to the possi- 
bility of changing the higher-level tacit assumption that the “rule” must be 
some number sequence, with letters and all other symbols excluded. The 
decision maker proposes 5 (the result of the prime number sequence) or 6 (the 
result of a Fibonacci set wherein each digit is the sum of the preceding two 
digits). Only when generalizations at this level of probability have been ex- 
hausted (within the domain of whole numbers) will the reasonable decision 
maker question the higher-level tacit assumption. Could it be that the given 
sequence of numbers was taken from an automobile license plate? Possibly 
the generalization should include letters of the English alphabet. 

A heuristic program, then, requires the automaton to hold some se¢ of gen- 
eralizations as a solution to each inductive decision, even though its direct 


solution may be the result of only one member of this set. The set of gen- 


eralizations covers various levels of abstraction; those at the higher level are 
usually carried forward to guide the reasonableness of the succeeding (and 
more probably successful) approach to the problem area. 


These concurrently generated rules may each be taken as a model of that © 


transduction required by the real world environment at its own level. As 


such they represent a set of normative self-images by the automaton. In life — 


the rule that is actually drawn upon may not always correspond to the lowest — 


level of abstraction. The organism might more naturally choose to utilize 
that level of abstraction at which the normative model and the real world are 
already almost in agreement, thus requiring a minimum of change. The simi- 
larity of such behavior to that of subjects under psychoanalytic therapy is 
obvious. The initial change of behavior serves to maintain the present ego 
status (the normative self-model). ; 
There seems to be an underlying belief in scientific circles to the effect that 


the real world must be essentially simple. If a number of hypotheses fit the | 
data then it is the simplest of these that is considered most worthy of atten-.. 
tion. There are both good and bad aspects to this viewpoint. However that | 


discussion is considered beyond the scope of this paper, but may be cited in 


passing here as an example of the basis upon which the particular generaliza- - 
tion that is implemented can be chosen. Undoubtedly there are many other ' 
equally “rational” bases (note that this choice represents a governing decision , 
that forms a preprogram for the generated set of rules “that allow heuristic : 


reasoning”’). 
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To achieve a heuristic program, then, an automaton must have the capa- 
bility of developing a. number of models of its own behavior. This paper does 
not presume to summarize all aspects of heuristic decision making. Reference 
is made to the growing literature. !3+15.17,23 

Generalizations at various levels can be observed in the learning behavior 
of animals. As described by Hebb, it is possible to train both a rat and a 
chimpanzee to look for food behind a white triangle symbol (FIGURE 6A). This 
learning may take as many as 200, 300, or 400 trials for the monkey, while 
the rat may demonstrate the learned behavior after only 10 to 20 trials (or for 
somewhat more difficult discriminations, after 50 to 80 trials). Testing of 
these same animals with other stimuli reveals that their individual perception 
of the white triangle stimulus was indeed quite different. The rat offers only 
random response to FIGURE 6B, C, and D, whereas either F1GURE 6B or C will 
elicit a positive response from the chimpanzee, demonstrating generalization 
at a higher level of abstraction. It is interesting to note that the two-year-old 
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Ficure 6. Training symbol and stimulus symbols. 


human child will also recognize the training symbol in FIGURE 6B, C , and D. 
Possibly the set of generalizations may best be thought of as consisting of in- 
cluded subsets of the generalization at the highest level of abstraction. The 


- consistency of these subsets may determine the stability of evidenced behavior. 


Inductive decisions are most natural to all living organisms. The essential 


instinct for self-preservation leads to the desire for stability of the environ- 


ment, that is, cognition and recognition of all observed entities. Cognition 
requires classification and, as such, generalization. The result is induction. 


Projective Decisions 


The scientific method, as indicated in FIGURE 7, may be viewed as a struc- 
ture of the three syllogistic decision types. ‘Those aspects of the real world 
that form the subject of investigation are viewed with an inductive decision 
process as resulting in some semantic description in terms of a hypothesis or a 


‘model. This model is manipulated into some more tractable form called a 


solution, through the use of a deductive syntactic transformation. Finally, 


the particular point in question is brought into focus: estimate the condition 
of the subject system at a particular time or as related to the value of some 
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other parameter. This pragmatic link is traversed through an abductive de- 
cision using the solution as the given “tule.” The validity of prediction, post- 
diction, interdiction, interpolation, or extrapolation by the scientific method 
is dependent upon the error introduced in the triplet of decisions. 


Autonomous Decisions 


The autonomous decision reaches a higher level of logical complexity when 
an uninstructed automaton is released in some environment and finds it possi- 
ble to determine its own problem. All organized systems have a singular 
short-term property in common: the tendency to preserve their state of or- 
ganization. Living organisms clearly exhibit this self-preservation instinct; 
that is, they program their decision-making behavior to minimize the effect 
of a sensed threat to the environment and thus tend to maximize their proba- 
bility of survival. Self-preservation is common to all organizations of living 
creatures. For example, governments provide standing military forces, com- 
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FicurE 7. The scientific method. 
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munities have Chambers of Commerce, and social clubs take action for the 
maintenance of their existence as any threat appears. (The citizen’s secondary 
instinct of procreative reproduction may be viewed as the primary self-pres- 
ervation instinct of the society of which he is a member.) 

It is interesting to note that inanimate systems also exhibit a passive tend- 
ency to preserve their structural organization. Any mechanism is an organ- 
ized embodiment: its very organization provides a cohesiveness that resists 
destruction; that is, energy must be expended in order to dismantle or destroy 
even a nonliving system. Taking the broadest possible viewpoint, this passive 
resistance to disruption forms a kind of ‘‘self-preservation instinct.’ Such 
“structural static inertia” results from the strength of such factors as bonding 
of the component parts, the information flow within the system, and the _ 
amount of memory. 

Autonomous decision making requires the active pursuit of self-preservation. © 
To accomplish this'end the automaton must hold, let us say, some degree of 
awareness of its own operation, or some self-recognition. This can be accom- 
plished if the automaton can generate a model of itself. For example, a model 
of only the spectral properties of the basic transduction is indicated in FIGURE 
8. A transformation can be applied to both the reference and controlled 
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variables to yield their spectra. These are compared to form a model of the 

automaton consisting of a rational fraction polynomial that expresses the am- 

plitude versus frequency behavior of the transduction. Under different cir- 

cumstances it might be more appropriate to generate a statistical model of the 

transduction through the use of tabulating translators permitting the genera- 
tion of a stochastic representation. 

Regardless of the technique by which this model is generated, it is never 
possible for the model to be an exact replica of the transduction process. 
There is always some delay in translation and, in the statistical model, the 
essential assumption of even short-term stationary status prevents the corre- 
spondence from ever being complete. Any noise introduced immediately 
corrupts the model, thus obviating an exact representation. 

Humans behave in a somewhat similar manner in their use of models of 

themselves. Human mental models are certainly not an accurate representa- 
- tion; in fact, there is not even an attempt to provide full representation. The 
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Ficure 8. Generation of model of transduction. 


human wants a simple “‘picture”’ of his own operation in order to permit simple 
reasoning in the face of a particular aspect of his environment. It would 
admittedly be inefficient to approach each task at hand subjectively with a 
complete representation of all aspects of one’s self, even if this could be at- 
- tained. : 

fm. Let it be presumed that some portion of an automaton is devoted to the 
generation of a model of its over-all transduction. As time proceeds this 
~ model will reflect the degrading effects of noise, both internally generated and 
~ externally incident disturbance. With this model available as evidence of 
- this “real world” situation it is possible to superimpose an optimizing deduc- 
4 tive decision process wherein either the initially programmed (genetically 
_ given) ideal model of the transducer or some recall of a previous model state 
~ is used as the reference signal. The result of this comparison of the reference 
" signal to the model of the actual transduction in itself forms a new model of 
the difference in level of self-preservation. This higher level model provides 
a separate supervisory function that monitors the degree of system success 


_ with respect to survival. i 
 Ficure 9 indicates such an automaton. Many other possibilities exist, 
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including the comparison of the actual transduction (as represented by the 
model) to a prédicted model that forms the reference signal. The prediction 
could be carried out through a rather complex decision’ procedure that is se- 
verely sensitive to the relative conservatism or radical attitude in terms of © 
the relevant noise. f 

It may be speculated that the essential difference between man and the 
lower forms of animal life lies in his capability for apperception. A multiple 
structure of models allows him to think about what he is thinking about, and 
it is felt that herein lies the key to abstraction. It seems reasonable to pre- 
sume that an inanimate automaton could be designed to include models at 
various levels and in such a way accomplish an analogy to thinking about 
thinking. The cortex of the brain is well known to form an analogical model 
of the human sensory-motor transduction. Furthermore, additional related 
models have recently been identified within other portions of the brain. Could 
it be that the identification of such a sequence of models might yield insight 
into the inherent human thought capability? 
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FicureE 9. Autonomous decision. 


The key to achieving autonomous decision making may prove to be the 
separate use of a portion of the decision-making system to form a comparative 
model of its own change of status. It is on this basis that the program of the 
basic transduction may be modified to prove more efficient with respect to 
the encountered environment or a change of criteria. It is the noise acting - 


on the automaton that furnishes the motivation for higher levels of perform- 
ance. 


The Decision “Continuum” 


In the human operator it is indeed difficult to determine discrete levels of 
complexity in decision making. An inexperienced driver may accomplish | 
steering control at the moronic level and white-line tracking by deductive 4 
optimization. As experience accrues the tracking task becomes more and 
more one of conditioned reflex until it too finally reaches the less complex 
status of a moronic transduction. It is generally true that the learning process 
succeeds in replacing higher levels of complexity with those of lower level to 
accomplish the same required result. More complex decisions of higher 
logical complexity are called upon less frequently. However, the result of 
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their action may be expected to be more dramatic, and their occurrence is 
therefore more obvious. There appears to be no reason to believe that an 
upper limit of logical complexity has yet been reached by the human operator 
in his decision making. 

In 1936 Turing” succeeded in specifying a “universal machine” that pos- 
sessed but few internal states and external transduction abilities. However, 
it was shown that such a Turing machine could be programmed to reproduce 


_ exactly the behavior of any physically realizable machine as long as no limita- 


tions were placed on the required time or memory capacity. Human operation 
is of some finite, though large, number of internal states. It therefore follows 
that in theory, at least, it should be possible to duplicate exactly human de- 
cision making provided, of course, sufficiently detailed information was made 
available describing the human transduction. Such a programmed Turing 
machine might even exceed the human capability with respect to equivalent 


_ logical complexity. Computers have already performed certain heuristic 


problem solving for which no algorithms are available and reprogram them- 


_ selves (learn) only on the basis of experience. 


The Intelligence of Decisions 


In 1873 Jevons® stated ‘“Economy of mental power may be considered one 


_ of the main conditions on which our elevated intellectual position depends.” 
- Intelligence may be defined as the absolute probability that a proper transduc- 


tion will occur for any particular given task. The more efficient the automa- 


4 ton is in time and effort, the greater will be this probability (reliability). It 
- follows that intelligence of decision transduction must be between zero and 


y 


unity for any task (defined in parameters and time tolerance). Perfect intelli- 
gence is unrealizable due to the uncertainty caused by real world noise. Zero 


+ intelligence is ruled out by the pure chance possibility of any desired occur- 
rence. Even completely random unstructured automata might happen to 
_ create the required output that corresponds to an acceptable solution. This 


* 


2 


fd 


_ probability, granted, is extremely small, but it always remains greater than 
_ zero. Specific calculation techniques are available from work done in statisti- 
~ cal mechanics!*.?° and stochastic processes.”! 


The above-described ordinal scale of logical complexity of automata may 


be viewed as a set of alternate hypotheses available to solve the problem at 
hand. It may be possible to choose too simple or too complex a program. 


In fact, certain problems may not be solvable below some level of logical com- 
plexity. Only the optimal level of complexity provides the most intelligent 


decision making. ; 
Human intellect appears limited by the efficiency of self-programming and 


reprogramming. Today this property can be measured only in terms of the 


intelligence of accomplished decisions. The near future will bring increased 


unity of the human operator and his computer tool. As a result the human 
may achieve greater reliability of decision than was previously possible. Is 


it not possible that the self-reprogramming of a computer could prove to be 


even more efficient in choice of program and reliability of execution—and thus 


far surpass the intelligence of its human counterpart? 
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Practical determination of the intelligence of any particular decision may 
prove to be a difficult task. The variability imposed by such factors as noise 
and transduction change by learning or fatigue all combine to prevent exact 


reproducibility. Only through recourse to statistical techniques is it possible — 


to estimate the desired probability of correct solution. The larger the sample 
of available experience within a particular class of decisions, the more accurate 
will be the estimation of individual decision intelligence. The simple corre- 
spondence of intelligence to decision reliability makes it possible to utilize 
well-known analysis techniques. Complex systems may be reduced to func- 
tional schematics and the related reliability equations. In the case of man- 
machine systems, machine reliability may best be interpreted in terms of an 
‘“ntelligence level.” This correspondence may offer new insight into optimal 
synthesis of future man-machine systems. 


Conclusion 


Man is increasingly dependent on the technology he has created. Devices 
and techniques that originate as luxuries rapidly change into necessities. Ex- 
panded suburbia, for instance, makes the automobile an essential part of life. 
Job assignments become more and more specialized by using the “‘tools” that 
extend human capability. These reach out in every direction, including ampli- 
fied force, greater communication range, and faster transportation. Not the 
least of these relates to data processing, allowing the replacement and realloca- 
tion of the human mental capability. Progress toward automation is no longer 
bounded by the limitation of the available hardware; rather, the design of 
efficient complex man-machine systems rests upon the proper determination 
of assigment for each of the decision-making elements. 

To permit this advance it is considered essential to provide a set of adequate 
descriptive measurements for the decision tasks to be accomplished through- 
out the system. It has been the purpose of this paper to offer but one such 
measurement with respect to the often used (and misused) term “‘intelligence.” 


It is not intended to imply that the proposed quantitative scale offers the sole _ 
valid approach. There may indeed be many useful techniques for measure- 


ment that deserves extensive exploration. If this paper serves to stimulate 
inquiry in that direction it will have performed its mission. 

The near future will see an even closer relation between man and machine 
in the form of a “man-computer symbiosis.”2 The human operator will de- 
pend upon the computer to a large extent as an increase in mental capability. 
The machine will be referred to for technical manipulation, memory and in- 
formation retrieval, language translation, and many forms of complex problem 
solving. Such man-computer teams might exhibit decision-making behavior 


at higher levels of intelligence than can be attained by man alone. Is it — 


possible even to explore this realm of intelligent behavior from the limited 
vantage point of the present? 


Considerata 


The above discussion leads most naturally to certain fundamental questions 
that deserve specific consideration. It would appear that a noiseless decision 
maker can certainly contribute no new useful information, even though it 


—_ 


Fogel: Intelligence in Decision Making 749 


might yield decision outputs of much greater complexity than the given data 
at the time for decision. Thus additional information may be a product of 
the previously collected data, as stored in a memory or the essential structure 
of the device. That is, it may even be possible for the decision-making autom- 
aton to convert some of its own complexity into the output solution at no 
cost to the structure if a nondestructive readout is available. The term “new” 
information then must be considered to include only information that has 
never been available to the decision maker, including its own generation-de- 
sign. 

Let us now consider the effect of noise introduced during the decision-making 
process. As indicated in the above section, the extraction of useful informa- 
tion can be achieved through the action of a suitable filter devised to interpret 
properly the given set of criteria constraints required for successful decision. 
It is possible to challenge this formulation by saying that there is really no 
“new” information, since the filter is truly adding the information by selec- 
tively operating on the random noise. It is the selection of useful versus non- 


_ useful pieces of random data that truly contributes the added information. 


Upon this basis the structure (of the filter) is contributing the additional use- 


ful information. In a sense, this point is analogous to the problem of Max- 


well’s Demon, which contributes the energy of selection in the famous informa- 


~ tion theory problem. 


Is it possible to find a selection filter that could operate without having been 
programmed for the criterion of success? Stating the question in another and 
more general way: — Is it possible to find a device that will classify without being 
programmed to do so? Since any inhomogeneity within the device may re- 


sult in a separation of random data, it is necessary to start with the concept 
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of a completely random structure. Only such a “device” can be said to be 
truly unprogrammed. The molecules within the system are excited by the 
ambient temperature, but in no way do they indicate any individuality. 

At any particular point in time the fluctuations within the structure prevent 
its perfect inhomogeneity and, as a result, the device may selectively filter 


_ the inherent noise and effect the output solution. Such filtering would not 
' prove useful since the particular nature of the filter would change from mo- 
~ ment to moment in a random manner. The problem must, therefore, be re- 


stated: Is it possible to find an initially random structured device that will 
attain and retain the ability to filter selectively? Is there the possibility of a 


“randomly organized structure becoming classified (structured) within itself 


and retaining that particular structure? If the answer is in the affirmative, 
then induction (which results from comparative classification) may truly be 
achieved by an inanimate machine. If the answer is in the negative, induc- 
tion can be accomplished only on the basis of some requisite programming of 
a machine that instructs the set of classes of concern or some primitive set 
of classes from which the machine can deductively derive the desired set for 
the immediate problem at hand. 

In order to answer this question it is necessary, first, to define more carefully 
the term “retain.” For how long must the structure be other than random 
in order to prove a useful selective filter? Certainly the happenstance of 
classes must last for at least the time required by the decision process. If, 
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however, there is no a priori knowledge of when the decision will be called for, 
it would be necessary for the classifying structure to tend toward some sta- 
bility. That is to say, this requirement would be satisfied only by a device 
that tended toward a particular ultimate classed state, but this appears to be — 
in direct conflict with the second law of thermodynamics, which indicates that ~ 
all systems will tend toward maximum entropy, toward greatest homogeneity. 

This “law” of thermodynamics must be viewed with caution. It states 
that the system’s increase in entropy is a statistically defined trend, but there 
is no requirement, for example, for monotonicity or for internal irreversibility 
of energy exchange. One of the defining characteristics of a living system is 
that it decreases its entropy; that is, it increases its structural complexity at 
the cost of a less-structured environment. Furthermore, even a nonliving 
system can have subsystems that, for some limited period of time, may oper- 
ate in a completely reversible manner. 

Returning to the basic question, it would seem that generally no mechanistic 
induction is possible (unless the entire system were living). The statistical 
dictates of the second law of thermodynamics in no way prohibit life, but they 
do place statistically flexible, finite bounds on the time period that may be 
used. In the specific instance of the automaton system it would appear pos- 
sible to achieve classification without special programming for sufficient time 
periods to allow useful results. A living organism, for example, may fall into 
this class. This particular state measures zero over the infinite time domain. 

There still remains a fundamental point. Only a “useful” selection filter 
is of any value. The probability that the random retained structure will 
prove to be “just right” is of negligible value. Only some communication 
from outside the automaton can offer the required guidance to avoid undirected 
induction (the extraction of useful data from noise is itself truly an inductive 
process, even though this was subsumed under a gross category of deductive 
decisions in the section above for a tutorial purpose). External communica- 
tion can take place only through the identification of a class system (since this 
is a prerequisite for any communication system). Such instruction might 
just as well dictate the structure of the filter and not depend upon the availa- 
bility of any self-structured state. In this case the induction is once again 
truly that of the external “instructor” and not of the filter, and the argument 
has returned to its starting point. | 

To summarize at this point: the logical process of an automaton can be dis- 
cussed only if the system that is the automaton is clearly delineated. The 
philosophical inquiry touched upon should not impose any impediment to the 
use of the indicated classes for the description of decision making. It is felt 
that such identification serves a useful purpose in directing the mathematical _ 
inquiry regarding such factors as effectiveness, efficiency, and intelligence to | 
facilitate optimal use for the available decision-making automata. 

The purpose of this discussion has been fulfilled if the fundamental questions 
are made apparent. Too often such basic inquiry is avoided on fear of tread- 
ing on the “metaphysical” or of opening controversy. It is only when igno- 
rance is discussed that any new contribution can result. Avoidance of un- 
certainty prohibits progress. 
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PHYSIOLOGICAL IMPLICATIONS OF 
PSYCHOPHYSICAL DATA* 


Wilson P. Tanner, Jr. 
The University of Michigan, Ann Arbor, Mich. 


Introduction 


In this paper I am presenting an approach to the study of psychophysics 
that includes the use of mathematical models. While the individual experi- 
ments can each be interpreted as a contribution to an area in psychophysics, 
the program can be interpreted as an experiment testing the feasibility of a prof- 
itable use of a particular scientific philosophy in the study of an area of psychol- 
ogy. 

This philosophy involves the use of mathematical models in a manner similar 
to that of the physicist. It employs, first, the use of simple models describing 
the relations of only a few observable variables. The measures describing the 
relations among these variables may be measures of variables that exist only 
as force and gravity in physics and that can be measured only by observing 
their consequences. As measurement techniques become more nearly precise 
or, as the conditions of the experiment are extended to include a wider range, 
it will likely become apparent that the model had been adequate only because 
it was tested with coarse measurement over a restricted range of conditions. 
At this point it will be necessary to expand or modify the model to include the 
new data. 

The expanded or modified model should still include the original model as 
applied to the limited conditions. FicGuRE 1 is intended to illustrate this 
point. The large space is one that includes all possible data that might be 
relevant to a particular area of science. Many of the points in this space will 
never appear as data for they are contradictory to the “truth,” whatever that 
may be. The shaded area in the center represents the data at hand. A model 


describing these data could be expanded in any of a number of directions. © 


The large areas A, B, and C represent models, each of which includes the origi- 
nal model. If the next datum collected turns out to be at point 1, model C 
can be eliminated from consideration. If it is collected at point 2, A and C 


can be eliminated. Either point, however, requires an expansion of the origi- 


nal model. 


We have tried to work with a general model that includes all of the points 
in the space. It is the general model of testing statistical hypotheses. The 
model can be expanded to specify the data that must appear if a sensory 
threshold exists, if Luce’s* axiom 1 is valid, or if the observer is intelligent. 
The general theory can be made specific in a number of ways; it is an experi- 
mental question that determines precisely how it should be made specific. 

I shall trace one phase of the history of this program, showing how we in- 
troduced the concept of memory to modify the model to incorporate a wider 

* The work described in this paper was supported in part by the United States Air Force 


under Contract No. AF49(638)-369, monitored by the Office of Scientific Research of the 
Air Research and Development Command, Wright Air Force Base, Dayton, Ohio. 
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range of conditions. At times I shall try to point out some of the physio- 
logical implications. However, one must not forget that I am talking about 
concepts that are unobservable, and the physiologist may find himself faced 
with similar problems to those of the physicist chasing electrons: he may have 
to devise physiological cloud chambers so that his electrons will create an ob- 
servable phenomenon. The nerve impulse may not be sufficient. 


Ficure 1. - Theory space. 


SIGNAL RESPONSE 
TRANSMITTER ENSEMBLE 


ENSEMBLE 


CHANNEL 


Ficure 2. Block diagram of a typical psychophysical experiment. 


What is a Psychophysical Experiment? 


Psychophysics is the study of the relations of psychological quantities to 
physical quantities. A block diagram of a typical experiment is shown in 
FIGURE 2. The experimenter designates an observation interval in time, dur- 
ing which one of the signals from the ensemble is transmitted over the channel. 


_ At the end of the observation interval, the observer is asked to choose one of 


SEO ep ee 


a set of responses. The joint event of the signal transmitted and the signal 
received (as indicated by the response) is recorded. 
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What are the Measures? 


In the simplest type of psychophysical experiment, in which a particular 
signal or nothing is transmitted during the observation interval, the data may 
be summarized as illustrated in FIGURE 3. In this figure, the rows represent 
the transmitted events as modified by the channel. If the signal is trans- 
mitted, the observer’s input is the signal as randomly perturbed by the chan- 
nel. Here we are calling this input ‘“‘signal plus noise.” If nothing is trans- 
mitted, the observer’s input is labeled “‘noise.” 

The columns are the observer’s responses. ‘Yes’? means that he considers 
the input to have arisen from signal plus noise, while ‘““No” means that he 
considers the input to have arisen from noise alone. The data can be sum- 
marized as percentages that, in analysis, are assumed to be estimates of proba- 
bilities. Two conditional probabilities are estimated: the probability that if 
the signal is transmitted the observer’s input will lead to a “‘yes” response, 


YES NO 
SIGNAL n, 
PLUS BulA) = “n+ ny 
NOISE 
NOISE it ns 
FSA) = n,+n, 


Figure 3. Data summary of a typical psychophysical experiment. 


Psy(A); and the probability that if nothing is presented, the observer’s input 
will lead toa “yes” response, Py(A). Psy(A) is the detection probability 
and Py(A) is the false-alarm probability. 

In FIGURE 3, 


Ny 


Psy(A) & Py ape 
BT 2 


and 


Px(A pp. 
w(A) ree 


where the n’s represent a counting of the various joint events occurring during 
the experiment. 

The fact that we summarize the data as estimates of probabilities implies — 
that the tasks the observer is asked to petform in psychophysical experiments 
are tasks in which there is a probability of error; consequently they satisfy 
Middleton’s criterion (personal communication) as tasks that can be handled 
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within the framework of decision theory. According to this criterion, when- 
ever one is confronted with a probability of error he is confronted with a de- 
cision theory problem. i 


Decision Theory Model 


Decision theory serves as a foundation for the theory of signal detectability, 
a theory that deals with exactly the problem with which the observer is con- 
fronted in a psychophysical experiment. An examination of the general theory 
of signal detectability may help us understand the observer’s problem. 

The problem handled in this theory is that in which an observer is asked to 
state which of two conditions led to a particular input, x, during a fixed ob- 
servation interval, 0 to T. Any input x is considered to be a function of time, 
a(t), since time is the dimension of the observation interval. With each x(é) 
two numbers are associated: the likelihood of that particular x(t), if the signal 


_ was transmitted, fsy[x(¢)]; and the likelihood of that particular x(t), if nothing 


_-that 


was transmitted, fy[x(¢)]. The decision theory problem is to select a criterion 
space, A, which includes all of those «(¢)s that should lead to the response 
“yes” and excludes all of those x(¢)s that should lead to the response “no.” 

This can be stated as the problem of choosing the criterion space A such 


Psy(A) — WPy(A) = a max (1) 


_ where W is a number or weighting function. 


The following proof, derived by Fox,! demonstrates that the likelihood ratio 


describes the information contained in the input, «(é), relevant to the choice 


the observer must make. Equation 1 can be written as 


fae dx(t) — WJafylx(d)] de) = a max. 


Since both variables are over the same space, this becomes 


J font d — Wfrlx()] de(t) = a max. 


In this form it becomes obvious that the integral is a maximum if all posi- 


“tive values are included and no negative values are included. Thus the space, 


A, should include all of those values of x(¢), and only those values, which 
satisfy the following inequality: 


fev(x()] — Wale] = 0 


or 


- where J[x(t)] is defined as the likelihood ratio. 
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What is an Ideal Observer? 


Given Fox’s proof, we are now in a position to define an ideal observer, illus- 
trated in its general form in FIGURE 4. It is an observer who computes the 
likelihood ratio associated with the input, «(¢), and then matches this value to 
a point in the criterion space. If the point is contained in A, then it says 
“yes,” the signal was transmitted. If it is not in A, then it is in the comple- 
ment of A, CA; and the observer says “no,”’ the signal was not transmitted. 
The computation of /[x(¢)] and the matching are performed without error. 

If we were to attempt to designate in detail the construction of an ideal 
observer, it would be necessary to know the signals that might be transmitted, 


LIKELIHOOD] 3 
RATIO 
COMPUTER 


DECISION 
COMPUTER 


TRANSDUCER 


Ficure 4. Ideal observer. 
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FicurE 5. The ideal observer with tie-ins to the system. 


LIKELIHOOD 
RATIO 
COMPUTER 


the properties of the perturbing noise, the way in which the signal and noise 
are combined, and the function W, to be maximized. All of these parameters 
are required to specify the specific components (such as filters and integrators) 
and the particular criterion spaces, A and CA. Any change in the values af 
parameters can lead to a change in the detailed designation of the ideal ob- 
server. Thus if we want to consider the ideal observer as applied to specific 
situations, then there is an ideal observer for each specific case. They are all 
alike in that they compute likelihood ratios and say “yes” if the likelihood is 
greater than W. Whether any specific x(¢) leads to a “yes” or a “no” response 
depends on the signal and noise conditions from which it might have arisen — 
and on the particular function, W, used in the maximization. : 

The whole operation is illustrated in FIGURE 5. The part of the block dia- 
gram included in the dotted lines can be considered as an ideal sensory system. 
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Those parts outside the dotted lines are the connecting links between the 
sensory system and the larger system that the sensory system serves. The 
connecting links mold the sensory system into an ideal form for the particular 
task at hand. It sets the parameters of the components and specifies the 
criterion spaces in a way that leads to optimum performance. 


The Human Observer and the Ideal Observer 


We probably all agree that, in the strict sense of the word, the human ob- 
server cannot be considered as an ideal observer. Few of us would expect 
perfect computations of likelihood ratios and perfect matching into the cri- 
terion spaces. 

However, we might feel that the human observer’s sensory system is, in a 
broad sense, similar to that of an ideal observer in some respects. It is, after 
all, a subsystem of an “intelligent” system and it seems not unlikely that the 
intelligent system would have some control over the form the subsystem as- 
sumes in facing any particular task. It may be molded in a way such that it 
exhibits a phenomenon we frequently refer to as “attention.” Its performance 
may vary in a way that is consistent with different methods for computing W. 
While it may not achieve an ideal performance, it may in some ways tend 
toward an ideal performance. 

If this is the case, then the experimenter must interpret his results in terms 
of the particular experimental conditions that have the potential of leading to 
a modification of the system. If these modifications are possible, then the 
physiologist is faced with a severe task in attempting to interpret data col- 
lected in experiments that eliminate the effects of the modifying agents. This 
line of reasoning clearly indicates physiological studies employing “‘con- 
sciously” behaving animals with implanted electrodes: the type of study I 


~ have heard Walter Rosenblith advocate many times. 


FicureE 6 shows a block diagram of an observer based on the ideal observer, 


but it also shows some of the places in which one might expect imperfections 


to exist. This diagram has as its basic structure that of the ideal observer, 
and the same links connecting the observer to the larger system. It has had 
added to it a number of memory banks that are immediately suggestive of 
imperfections. Let us examine the various properties an ideal memory must 


have in a simple detection experiment where the signal is a segment of a sine 
_ wave. This signal is specified exactly; thus when it occurs in the observation 
interval, 0 to T, it will occur at a precisely specified position within the inter- 


val. Its frequency, phase, and amplitude will also be specified precisely. This 


means that the function of time representing the signal can be positioned 
exactly in the observation interval. The value of this function can be pre- 
cisely stated for any moment of time in the observation interval. 


To be able to do this, an observer must have stored in his memory a precise 


a representation of the function of time. He needs a clock mechanism that 


permits him to synchronize exactly with the observation interval, 0 to 7, and 


~ to locate exactly in the interval the point at which the function of time should 
; start, and precisely its duration. His memory must specify exactly the fre- 
quency, phase, amplitude, starting time, and duration of the segment of the 
sine wave. Any error in the memory will lead to a decrement 1n performance, 
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It is possible to make some inferences from psychophysical experiments 
regarding the role that each of these aspects of memory plays in the human 
observer’s performance. It must be remembered, however, that these in- 
ferences are based on a model designed to describe the relation between the 7 
observables in a psychophysical experiment: the transmitted signals and the 
responses. The models can be looked at as mathematical equations fitting 
the data. As long as the data are finite, there are an infinite number of models 
that will fit equally well. Data collected in the future may eliminate this — 
model, but will never confirm it. It uses words such as “memory” for the 
parameters because these words have intuitive appeal. The physiologist 


1. SIG. PARA. 
2.NOISE PARA. 


Ficure 6. The ideal observer modified to include memory. 


trying to correlate his observations with the parameters of the model must . 
remember that the concept of memory arises from the model, not the data. 


A Psychophysical Investigation of “Memory” 


One of the concepts involved in using the theory of signal detectability as a 
mike interpreting psychophysical data is expressed in the following ‘“theo- 

If in two channels, one employing an ideal transmitter and an imperfect 
observer; and the other a transmitter with statistical uncertainty and an ideal 
receiver, both transmitters having equal energies available, the performance — 
as indicated by measures of the false-alarm rate and the detection rate are 
found to be identical, then the imperfect receiver can be said to introduce 


es oe degree of uncertainty in its channel as the transmitter in the other 
channel. 
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The uncertainty attributed to the receiver, it was felt, would reflect the ob- 
server’s memory. Consequently a series of models with specified transmitter 
uncertainties were studied, leading to calculations of the performance measure 
d’ as a function of 2E/N,. Ficurr 7 shows these plotted on log-log paper, 
and a human observer’s data superimposed. The notion was that if the hu- 
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Ficure 7. Performance curves for signals specified statistically. 


man observer’s data matched one of the curves, the degree of uncertainty 
introduced by him could be specified, and his memory characterized by a 
number. i 

The concept is far too simple. The human observer’s data did not fall on 
any one curve, but rather cut across curves. This suggested that a different 


- model would be required for every signal level, at least in.terms of the number 


4 ‘Y 


categorizing the memory. 
Careful examination of the experimental procedures under which the data 


_ were collected suggests that this is not unreasonable. Each signal level was 
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studied in a sequence of 100 observations coming about five seconds apart. 
Before the sequence, the observer was permitted to listen to several samples 
of the signal without noise, and these noiseless samples were assumed to es- 
tablish his memory. 

If the signals are weak, then, as the sequence proceeds, the observer will 
receive few samples that are clear enough to reinforce his memory. If they 
are strong, he is more likely to receive some fairly good samples, and his mem- 
ory on the average should be better throughout the sequence than it would 
be for the weak signals. 

Elizabeth Shipley (personal communication) has recently performed an 
analysis of some data in which she considers the probability of a correct choice, 
given that the previous choice was correct and given that the previous choice 
was incorrect. She finds that a dependency exists when the signals are seg- 
ments of sine waves, and does not exist when the signals are samples of noise. 
The dependency is in the right direction to support the hypothesis that a clear 
sample tends to reinforce the memory. 

When Julian Bigelow saw some of the early results, he suggested a way of 
attacking the problem experimentally. He suggested that a sine wave of the 
same phase and frequency as the signal be added as a component of the noise, 
replacing the need for the observer’s memory of frequency and phase. The 
signal would be superimposed on the sine wave during the observation inter- 
val. This procedure altered the slope of the psychophysical function so that 
it agreed with one of the slopes of the computed curves of FIGURE 7. How- 
ever, it is positioned wrong. 

There was still a need for the observer to have a memory for starting time 
and duration. My colleagues and I were able to extend Bigelow’s suggestion 
in a way that could remove the need for this memory. This we have referred 
to as a “pulsed carrier” experiment. In each observation interval we intro- 
duced a segment of sine wave whose amplitude was large. This segment 
matched the signal in frequency, phase, starting time, and duration. It was 
the observer’s task to state whether the signal appeared on top of this pulsed- 
carrier or pedestal, as J. C. R. Licklider calls it. In this case, the psychometric 
function turns out to be linear, or as if there were some added noise. 

This leads to an interpretation of the memories required as a function of the 
experimental design, as illustrated in TABLE 1. There are three types of ex- 
periments: detection, continuous wave, and pulsed-carrier; each with two 
subdivisions, “‘yes-no” and 2AFC. The detection experiment requires a mem- 
ory for frequency, phase, starting time, and duration. The CW experiment 
requires only starting time and duration memories, while the pulsed-carrier 
experiment requires none of these. The difference between a “yes-no” experi- 
ment and a 2AFC experiment is the need for amplitude memory in the “‘yes- 
no” experiment. 

A summary of the results of the various experiments follows: 

(1) In the detection experiment, the slope of the psychometric function is 
too great to be accounted for in terms of a fixed uncertainty. There is no 
difference between a yes-no” and a 2AFC experiment, probably because the 
other memories are so poor that they mask the effect of amplitude memory. 
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(2) In the CW experiment, the slope of the psychometric function is such 
that it can be accounted for in terms of a fixed uncertainty. The function is 
misplaced, prohibiting an estimate of the degree of uncertainty. There is 
little, if any, difference between “yes-no” and 2AFC experiments. 

(3) In the PC experiments, d’ is a linear function (2E/No)!2, a condition 
that can be accounted for by additive noise. Performance in 2AFC experi- 
ments is noticeably superior to that in “yes-no”’ experiments. 

The analysis of the memory ccnditions shown in TABLE 1 is not precisely 
true for an imperfect observer, who for example must have some amplitude 
memory in the 2AFC, PC experiment and must be able to remember the am- 
plitude of the waveform in the first interval so that he can compare it to the 
amplitude of that in the second interval. In fact, the experiment can be 
looked at as a “‘yes-no” experiment in which the measure established in the 
first interval serves as a cut-off for the second. If the intervals are widely 


TABLE 1 
MEMORIES REQUIRED FOR DIFFERENT PSYCHOPHYSICAL EXPERIMENTS 


Required memories 
Experiments ; 
Amplitude |Frequency Phase prone Duration 
‘ “Yes-No”’ deat aa <= ae = 
Detection 2 AFC ae ae ap an 
Conti “Yes-No”’ ai st ae 
ontinuous wave 2 AFC ss sk 
i “ Ni ”? 
Pulsed carrier eae + 


separated, this cut-off would not be precisely remembered. It might be as if 
noise were added to this cut-off, the amount of noise depending directly on 
the time between the two pulses. 

‘Cut-off variance can be treated as additive noise, provided that all of the 
prior stages are linear, as illustrated in FicuRE 8. There are three distribu- 
tions shown: in each the abscissa is the value of the decision variable, and the 


ordinate is the probability density of that variable. The distribution to the 


left is for the second signal, given that it is the smaller of the two. The dis- 


- tribution to the right is for the second signal, given that it is the larger. The 
_ middle distribution is the critical value of the decision variable, «, . 


It is possible to transform to a new decision axis, « — x, , where on this new 
axis there is a fixed cut-off at 0. There are now only two distributions, « — x, 
given the smaller signal, and x — x, given the larger signal. Each of these 


has added the variance of x, to its original variance. 


Using the form of the psychometric function for the pulsed-carrier to dem- 


onstrate that the stages prior to the cut-off are linear, it should now be possi- 


ble to study the variance of the cut-off as a function of the time, 7, between 
the observation intervals. One complication still exists, however: there must 
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be a transducer type of operation at the input and, if T is too small, inde- 
pendent observations cannot be expected. Thus the performance should drop 
both for very small values of T and for very large values of JT. In between 
there should be a peak. In fact, the following equation should specify d’: 


2Ea 
No + kT 


where e is something like the time constant of the transducer, & is the noise 
energy per unit time added to the cut-off value, and 7 represents the efficiency 
if added noise of the nervous system alone accounts for the fact that the effi- 
ciency is less than unity. A typical set of results is shown in FIGURE 9. 


(d’)? = mo(4 BaP 


x = CUTOFF VALUE OF x 


X— Xe 


Ficure 8. Illustration of the cut-off variance as added noise. 


The CW experiment, using the same observer, may be re-examined now. 
Suppose it is assumed that the value of 2H/No used in the experiment has 
been reduced to the value of (d’)?, as calculated by the equation with the pa- 
rameters determined by the pulsed-carrier experiment. One may now plot 
the psychometric functions as (d’)? versus the corrected value of 2E/N (FIG- 
URE 10). The misplacement of the curve is now corrected so that it is possi- 
ble to estimate the degree of statistical uncertainty introduced by faulty mem- 
ories of starting time and duration. The values of this parameter for the four 
observers range from about 5 to approximately 12. 


All of the experiments thus far discussed have involved acoustic signals. My | 


colleagues and I have also made visual studies, and R. Fitzhugh? at the Johns 
ees University, Baltimore, Md., has reported some physiological data 
related to these siudies. At first appearance there seems to be a discrepancy 


. ’ | 
between Fitzhugh’s data and ours. However, the discrepancy disappears when 


it is considered in terms of the memory framework. 
Fitzhugh studied the frequency of nerve impulses on the dark-adapted cat’s 
eye, counted over an arbitrary time interval after a flash of light had been 


~ 
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FicurE 9. Efficiency as a function of amplitude memory time. 


764 Annals New York Academy of Sciences 


100.0 
SIGNAL SPECIFIED EXACTLY——_, 
10.0 2 x 
/ 
/ 
re) x! 
(a')* / 
/ 
Vis ; 
(@) ras 
Ss 
, FA DATA 
me) 
/ 
/ 
/ 
o\ ri 
SIGNAL WITH UNCERTAINTY 
OF M=5 | 
x DATA 
© DATA CORRECTED FOR),,C, AND A 
0.1 | 
Ke) 10.0 100.0 
rag 
No 


FicurE 10 Data corrected to permit an estimate of starting time and duration uncertainty. 
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introduced. From his studies he was able to construct statistical distributions 
associated with each of a number of intensities (including 0 or noise alone). 
He could then assign values of d’ representing the separation between any 
one of these distributions and that for noise alone. According to Fitzhugh, d’ 
was approximately linearly related to AI, while in our psychophysical experi- 
ments d’ varied as AI”, where p was 2 or greater. An examination of our pro- 
cedure indicates that there might be uncertainty both as to the time and lo- 
cation of the signal presentation. The slope of our curves agreed with the 
slope of the curves for M = 100. The uncertainty may be introduced at a 
central rather than retinal level, suggesting that one should not try to account 
for nonlinearities at the level of the end-organ, to the exclusion of central hy- 
potheses. 

Qne more point on the relation between Fitzhugh’s results and ours. We 


_ analyzed our results on the assumption that normal distributions existed. 
- Fitzhugh’s distributions started at zero, and appeared quite skewed. This is 


not a discrepancy, for, from the standpoint of psychophysical analysis, any 
distribution that can be made normal by means of a monotonic transforma- 
tion of the decision variable can be analyzed in a fashion consistent with the 
normal assumption. 

To demonstrate this point, let us consider Fox’s proof, which shows that 


likelihood ratio is the proper decision variable. Likelihood ratio varies from 0 
- to infinity and, for noise alone, at least, is highly skewed. However, if one 
- considers the decision rule “say yes” if log, U(x) > log. W, every decision is 
_ the same as if it were made according to the decision rule ‘“‘say yes” if I() > 
_W. Log, I(x) varies from — 2 to +, and under certain conditions is nor- 
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mal. Since all the decisions are the same, and since one can only observe the 
decisions, a normal analysis can be applied whether the decision variable is 


* log I(x) or U(x). The results will be the same in both cases. 
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DECISION MAKING AND LEARNING UNDER VARYING 
CONDITIONS OF REINFORCEMENT* 


Sidney Siegel 
Department of Psychology, The Pennsylvania State University, University Park, Pa. 


From the work of philosophers, mathematicians, economists, and psycholo- 
gists, a theory of rational choice has emerged that considers the nature of ra- 
tional behavior in human decision-making.'-” 

According to this approach, when a person can specify a set of alternative 
outcomes such that for any strategy pi he knows the probability distribution of 
outcomes, that person is “rational” if he chooses a strategy that will enable 
maximization of expected utility. Most decision-making theorists take “prob- 
ability” to mean subjective probability, and “utility” to mean subjective 
value. Thus a strategy #; is admissible if there is no other strategy pj such 
that 


SEU (p;) > SEU (fi) 


where SEU (p) is the subjectively expected utility when strategy p is used by 
the person. 

Compared to the considerable amount of sophisticated theoretical work that 
has been done in connection with this theory of rational choice, there has been 
relatively little empirical work. However, some empirical work articulated 
directly with the theory has been attempted.?:°-" In addition, there is a con- 
siderable body of empirical work by psychologists that can be interpreted in 
terms of the implications of the theory of rational choice.'*-** However, when 
that theory is tested against the data presented in these studies, conflicting 
results are found. Whereas some of the studies support the predictions de- 
rived from a decision-making approach based on the theory of rational choice, 
others fail to support these predictions. 

The principal purpose of this paper is to bring psychological approaches to 
human choice behavior into more direct articulation with the theory of rational 
choice already stated. The balance of the paper is concerned with a class of 
psychological experiments involving choice behavior and learning: the two-| | 
choice uncertain outcome situation. Mathematical models and experimental 
tests of these models are presented. These will attempt to integrate the psy- 
chological approach with the theory of decision making. 


Stable-Stale (Asymptotic) Behavior in the Two-Choice Situation 


We are concerned here with an experimental situation that has been widely _ 
used in studies of human learning and choice behavior, a variation of which was — 
initially employed by Lloyd Humphreys.** The subject is seated before two 
lights and asked to predict which of the two will illuminate after a signal stimu- 
lus. On every trial, one and only one light illuminates. The two events oc-- 
cur with fixed but unequal probabilities, r and 1 — z, in a random sequence 


* The work described in this paper was supported i 
from the National Science Foundation, Waihinoten, D. an Py RE eee 
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for a number of trials. In some studies, devices other than lights have been 
used. In any case, the events are independent of the subject’s behavior: re- 
gardless of what he does, one of the two predetermined events will occur on 
each trial. The subject is instructed to do his best to predict correctly which 
of the two will occur. He expresses his prediction by pressing one of two but- 

_ tons. He is given no other verbal information about the situation, but he is 
allowed to witness the events and thus to determine for himself the correctness 
or incorrectness of each of his predictions. 

W. K. Estes has developed a statistical theory of learning"! that yields pre- 
dictions about an individual’s stable-state behavior in a situation of this sort. 
Estes’ model yields the prediction that subjects will learn to match their re- 
sponse ratios (the relative frequencies with which they predict each of the two 
events) to the actual probabilities of occurrence of the events 7 and 1 — rz. 

_ The same prediction is given by the Bush-Mosteller model?* when certain re- 
- strictions are applied to the parameters. A number of studies report findings 
in support of this prediction.!8-*4 

This is a curious result. Since the subject is instructed to do his best to 
predict correctly which of the two events will occur, and since we may suppose 
that he is attempting to follow those instructions, should we not expect him to 
learn to maximize the expected frequency of correct predictions? To do this, 

_ he should tend to predict the more frequent event on all trials. We might 
_ expect him to come to such stable-state behavior after an initial learning period. 
- Consider, for example, a two-choice uncertain outcome situation in which 
over a series of trials the two events, E; and E2, occur with probabilities 7 
and 1 — z of 0.75 and 0.25 respectively. Estes’ model asserts that a person 
will learn to make predictions that tend toward stabilizing at prediction of Fy 
~ on 75 per cent of the trials (6 = 0.75) and prediction of EZ, on 25 per cent of 
the trials. A person who adopts this ‘“‘strategy” will have 0.625 as his expected 
_ proportion of correct predictions, for 


i = pet ha — *) 
= 0.75(.75) + 0.25(.25) = (625 


On the other hand, if the subject were to adopt the “‘pure strategy”’ of predict- 
ing #, on every trial (p = 1.0), he would have 0.75 as his expected proportion 
of correct predictions, for 


wT 


ry 


E, = 1.0(.75) + 0(.25) = 0.75 


hay Sonia 


Because of the considerations sketched above, some behavioral scientists 
“have asserted that people who match their response ratios to the probabilities 
of the events are acting “‘irrationally” in that they are failing to maximize 
their expected proportion of correct predictions, a goal they could accomplish 
by predicting the more frequent event on every trial. The empirical fact is 
that in this situation most people do, after many trials, stabilize at matching 
their response ratios to the probabilities of E, and A,. To assert that this is 
“irrational” is to rely on a highly restrictive meaning of that term, It was 
“pointed out by Bernoullit in the first half of the 18th century that any theory 
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of rational behavior that does not incorporate the concept of the utility or sub- 
jective value of the outcomes rather than their objective value will lead to para- 
doxes of the kind under discussion. One must bear in mind the distinction 
between objective rationality (rationality as viewed by the experimenter) and 
subjective rationality (behavior that is rational, given the perceptual and 
evaluational premises of the subject). 

The purpose of the research to be presented here is to suggest, and demon- 
strate the validity of, a resolution of the apparent paradox. In this research 
a utility theory approach is incorporated. 

From decision-making theory®” a hypothesis of maximization of expected 
utility may be drawn that will account for both sorts of prediction behaviors: 
both matching response ratios and maximizing the expected frequency of — 
correct predictions. According to this approach, which has been formalized 
in two models to be presented later in this paper, whether a person will tend 
toward one or the other prediction strategy depends on certain conditions re- 
lated to the reinforcement inherent in the situation. The general hypothesis 
is that a person will behave as if he were attempting to maximize expected 
utility in any instance. When the components of the total available utility 
vary in magnitude from situation to situation, we may expect a person’s 
strategy to maximize expected utility to vary concomitantly. 

It seems reasonable to suppose that when a person is in a situation in which 
the only payoff or reinforcement from the outcomes is the satisfaction of seeing 
his prediction confirmed by the event or the dissatisfaction of seeing it discon- 
firmed, making a correct prediction of the rarer event has greater utility for the 
person than making a correct prediction of the more frequent event. The 
person derives satisfaction from ‘“‘playing a game with the machine,” trying 
to “outwit” it. 

Moreover, there is the matter of monotony, both kinesthetic and cognitive. 
Predicting the more frequent event on all trials, that is, utilizing the maximiz- 
ing strategy, would engender the monotony of pressing the same button on 
trial after trial for hundreds of trials. It would also engender the monotony of 
experiencing the same cognitive response, for example, “left button,” on trial 
after trial. 

Under such circumstances, a subject may maximize expected utility by using : 
a mixed strategy of splitting his predictions in some proportion. Matching 
his response ratio to the actual probabilities of occurrence of the two events 
would be one possible mixed strategy. The “cost” of an incorrect prediction 
is very low, whereas the gain this strategy may yield in terms of other utilities— 
such as the utility of gambling and the utility of variability (relieving monot- 
ony)—may be relatively high. Thus a mixed strategy may serve to maximize _ 
total satisfaction. 4 

If such an account is correct, then a decision-making model could yield the 
same predictions as the Estes model concerning the behavior of a person in a 
two-choice situation involving no special payoff. 

However, if the utility attached to correct predictions were somehow in- 
creased by a change in the conditions of the game, then the prediction yielded 
by the hypothesis of maximization of expected utility would diverge from that 
yielded by the Estes model. One may change the game so that there is poten- 
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tial satisfaction beyond that of having an event confirm one’s prediction, or so 


_ that the cost of an incorrect prediction is increased. If the decision-making 


theory approach proposed here is correct, the introduction of systematic varia- 
tion in the reinforcements (and thus utilities) attached to correct or incorrect 


_ predictions should be followed by systematic differences in subjects’ responses. 


As the utility of a correct prediction is increased and/or as the cost (negative 


_ utility) of an incorrect prediction is increased, the subject’s prediction of the 


more frequent event should tend to 100 per cent. That is, he may be expected 


_ to choose a “pure strategy” rather than a “mixed” one. 


In the section that follows a formulation of the decision-making theory ap- 


_ proach is given that is more precise than the informal comments above. This 
_ formulation, Model I, centers on two concepts: the utility of a correct predic- 


tion and the utility of variability. In Model I there is no explicit consideration 


_ of the utility of gambling. Thus, there is an implicit assumption—and this 
_ may be unreasonable—that when data on decision making consist of averages 
_ for groups of subjects there is little, if any, systematic effect from the utility 


of gambling, since this utility is positive for some people and negative for others. 


Model I 


Let + = the probability of occurrence of the more frequent event; 
p = the proportion of times the subject chooses the more frequent event; 
a = the marginal utility of a correct prediction; and 
b = the marginal utility of variability. 

If the expectation that a subject’s prediction will be correct, E; , is 


Be lpr kl = p= 2] 
= [(1 — =) + p22 — 1)] 


~ then the expected utility of a correct prediction, U,, is 


Nien, Yao NS Oh Shah SL SN Ba 


aN 


U, = aE, = al(1 — x) + p(2mr — 1)] 


If U, = the utility of variability = f(p), such that U, is symmetrical and 
maximal for p = 0.5, for example, p(1 — /), then 


U, = bp(1 — p) 
The total expected utility of a particular strategy p is 
U(p) aa U, =| U, 


= al(1 — 7) + p(2m — 1)] + bp — 9) 
The strategy p that maximizes expected utility is at 
dU(p) _ 9 
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If a = (a/b) 
then 


p=ar—)+ hs (1) 


It may be seen from 1 that when and only when a = b, and thus a = 1, 
p = x. That is, when the marginal utility of a correct prediction equals the 
marginal utility of variability, Model I makes the same prediction regarding 
asymptotes as the Estes model. This situation seems to hold when the only 
reward in the situation is knowing whether one’s prediction is correct or not. 

However, if the reinforcements inherent in the situation are increased, for 
example by adding monetary rewards and/or costs to the outcomes, and thus 
the utility of a correct prediction and/or the negative utility of an incorrect 
prediction is increased, then a > banda > 1. For such conditions the stable- 
state predictions yielded by Model I diverge from the predictions yielded by 
the Estes model. 

The predictions Model I yields concerning a subject’s choice and strategy 
behavior may be generally stated as follows: For any + 


(1) p=1 when a> [1/(2e — 1)] 
Qi> ps, whe le < (tri 
(3) p=3 when a=1 

(4) ><a when a<1 


The third prediction, that » = 1, is the prediction yielded by the Estes model, 
and is a special case of Model I. The first, second, and fourth predictions, it 
would seem, the Estes model is not prepared to make. 


Experimental Tests of Model I 


To date we have completed three independent experimental tests of Model I. 
In each of these experiments, the subjects were male undergraduate students 
and the experimenter was a male psychologist. Each subject was seen indi- 
vidually in a personal experimental session. He was seated before a board to ’ 
which a right and a left light plus a signal light were attached. He was in-_ 
structed that his task was to predict whether the right or the left bulb would 
illuminate after the signal stimulus. He expressed his prediction by pressing 
a button. Then he would observe which light in fact did illuminate. The two 
lights illuminated. according to a previously fixed random series, one illuminat- 
ing on © proportion of the trials, the other on 1 — x proportion. To control 
for possible position preferences, we used the right and the left lights as the 1 
more frequent ones for different subjects, at random, with 50 per cent of the 
subjects participating in sessions in which the left light was the more frequent 
one and 50 per cent participating in sessions in which the right light was the 
more frequent. The operation of the lights was controlled by electronic equip- 
ment, and the subject’s choices were recorded automatically. The experi- 
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menter was seated in the room in such a location that it was evident to the 
subject that the experimenter was not actively operating or tinkering with the 
a nor was he in any way determining which light illuminated on any 
trial. 
Each subject was exposed to a large number of trials (200 to 300) to give him 
_an opportunity to learn the situation and to reach a stable-state strategy. Our 
_ special interest in these three studies was in the asymptotic level of p (propor- 
tion of predictions of the more frequent light) that a subject would reach 
after many trials. 
In the various experiments we used up to three conditions of reinforcement 
_ in an attempt to manipulate the utility attached to a correct prediction and 
_ the negative utility attached to an incorrect one. The three conditions were: 
_ No Payoff, Reward, and Risk. 
_ Under the No Payoff condition, the reinforcement for each prediction con- 
_ sisted simply of seeing the outcome—whether the right or left bulb lit—and 
thereby determining whether one’s prediction was correct. This condition is 
_ the familiar and usual one in studies of human behavior in two-choice situa- 
tions. Under the Reward condition, the subject received five cents for each 
- correct prediction, the nickel being given to him immediately after each trial 
on which his prediction was confirmed. Under the Risk condition, the subject 
_ received a nickel each time his prediction was confirmed and forfeited a nickel 
if it were disconfirmed. Thus, under the Reward and Risk conditions, there 
was monetary reinforcement in addition to the reinforcement involved in 
_ determining the correctness of one’s prediction. ‘The two differed in that the 
_ Reward condition involved only increasing the utility of a correct prediction, 
_ whereas the Risk condition involved increasing the negative utility of an in- 
- correct prediction as well. 
_ Every subject, regardless of condition, was given a stake at the start of the 
_ experimental session, and told that whatever money he held at the conclusion 
_ of the session would be his to keep. 
_ First test of Model I. In the first experiment?’ 36 men served as subjects, 
and they were randomized into three groups. These were run under the three 
conditions already described: No Payoff, Reward, and Risk. For this experi- 
ment, r was set at 0.75. That is, one light illuminated on 75 per cent of the 
trials and the other illuminated on the remaining 25 per cent. The order of 
occurrence of the two events was randomized into blocks of 20 trials, and 
és subjects were run for a total of 300 trials. Every subject was given a stake of 
75 cents at the start of his session. 


_ The purpose of the experiment was to test a hypothesis drawn from the 


decision-making approach already introduced, a hypothesis formalized in 
Model I. Since the predicted strategy is 
p=amr—W)+ (1) 


3 where a = a/b, it was hypothesized that an increase in the value of a, achieved 


by offering an additional reward for correct predictions, would lead to an 
increase in p. 
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That is, using Estes notation,'® where Pi) is the mean asymptotic prob- 
ability of predicting /; for a group of like individuals, 


under no payoff, pi(©) = ™ 
under reward, pi(~) = m+ € 
under risk, P(o) =mte+tr 
The hypothesis, then, was that 
fi(o), under risk > (2), under reward > p:(%), under no payoff. 


The results confirmed the prediction at a highly significant level (p < 0.001). 
We determined the proportion of times that each subject predicted the more 
frequent light during the final 20 trials of his series of predictions. On the final 
20 trials of the first 100, these proportions were 


p (No Payoff) = 0.69 
p (Reward) = 0.78 
— (Risk) = 0.90 


By the end of 300 consecutive trials, when we may more safely speak of choices 
on the final 20 trials as representing asymptotes, the data were 


p (No Payoff) = 0.75 
p (Reward) = 0.86 
p (Risk) = 0.95 


That is, as predicted, subjects under the Risk condition will predict the more 
frequent light oftener than subjects under the Reward condition, and these in 
turn will predict the more frequent light oftener than subjects under the No | 
Payoff condition. Thus the data lend strong support to the contention from 
decision-making theory that a person behaves in this situation as if he were 
attempting to maximize expected utility. The utility hinging on correct or 
incorrect predictions differed under the No Payoff, Reward, and Risk condi- 
tions, and thus it seems that the attempt to maximize expected utility led to, 
different strategies under these conditions. ; 

Second test of Model I. The first test offered strong support for the general 
features of Model I. However, evidence was still needed to support its par- 
ticular features. That is, although the data supported the ordinal hypothesis 
that an increase in the marginal utility of a correct prediction, a, leads to an 
increase in p, the proportion of time the more frequent light is predicted, this 
finding cannot be taken as support for such particular features of Model I as, 
for example, that U, = bp(1 — p). This expression, although it has consider- _ 
able intuitive appeal as an expression for the expected utility of variability is, 
after all, an arbitrary one. Many other expressions could be used that would 
be consonant with the ordinal hypothesis supported in the first experiment. It 
was to test the specific mathematical structure of Model I that the second 
experiment was designed. 


For the second test”? 40 subjects were randomized into two equal groups, and 
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both groups were run under the Risk condition. The first group was run with 
m™, = 0.75, and from their data an & was estimated. This value was used to 
predict the value of p. obtained from the second group of subjects. These 
were run with m; = 0.65. All subjects were run for 300 trials, and were given 
a stake of 75 cents at the start of the session. 

Under Model I, @ is obtained from the data as follows: 


p=ar—4)+\% (1) 
and thus 
ae, Pin ye 
eas lo 


_ The_design of the study may be represented as 
™m =0.75 m2. = 0.65 


risk condition a 


predict 
po 


and the prediction was 
po= alm — 12) +e 


- ‘The results of this study offer strong support for the particular features of 
Model I. For the first group of 20 subjects run under Risk with +, = 0.75, 


Bi = 0.929 
_ and 
io . 
; Seen OEEs 
and thus the predicted value of p, was 
3 po = 1.718 (0.65 — 14) + 14 = 0.758 


When the second group of subjects was run under 72 = 0.65 and under the 
“same level of Risk as the first group, the subjects reached an asymptote of 
0.753 at the end of 300 trials. This value was remarkably close to the value 
"predicted from Model I on the basis of knowledge of the value of & in the first 


_ group: 
; predicted value of pz = 0.758 
4 observed value of 2 = 0.753 


. In addition to supporting the specific character of Model I, these data also 
. upport the notion that subjects will not match response ratios to input ratios 
when correct predictions are given increased reinforcements. Thus for the 
first group the input probability was 7 = 0.75, and the responses stabilized 
at ~, = 0.93. For the second group, the input probability was 72 = 0.65 and 
the responses stabilized at p2 = 0.75. These results are not unexpected in the 


ight of the findings of the first study. 
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In a later study, an opportunity arose to replicate this second test of Model 
I. Twenty subjects were run under Risk at 3 = 0.70. On the basis of the 
finding that & = 1.718 in the first study, we would predict for this third group 
that p; = 0.84. In fact, the responses stabilized at ps = 0.85, a value in close ~ 
agreement with the predicted value, and one that provides additional support ~ 
for the specific characteristics of Model I. 

Third test of Model I. The two studies summarized above support the 
general and specific features of Model I. That is, the data are evidence in 
support of the mathematical definitions of the variables of the model. How- . 
ever, they do not establish the validity of the psychological definitions of these 
hypothetical constructions. 

The expression in Model I for the expected utility of a correct prediction, 
U, = aE,, has considerable face validity and a sound logical foundation. 
Thus it was deemed advisable to focus attention on attempting to validate the 
variable that is perhaps psychologically more interesting, the expected utility 
of variability, U, = bp(1 — ). 

We may argue that if U, really stands for the utility of varying one’s re- 
sponses, then by giving subjects an opportunity to make the same prediction 
without being required to experience the same kinesthetic and cognitive re- 
sponses, we should reduce the marginal utility of variability, b, and thus raise 
a = a/b. Since p = a(t — 4%) + , an operation that increases a would 
increase p, the proportion of predictions of the more frequent event. Thus it 
is predicted that 


p (low 6) > p (No Payoff) 


The third test of Model I is based on the notion that the utility of variability 
can be reduced by giving the subject an opportunity to make the same pre- 
diction, for example “‘left light,” by different responses. We devised an ex- 
perimental situation that should operate to reduce the monotony engendered : 
by making the same prediction on trial after trial by providing opportunities _ 
for cognitive and kinesthetic variability in making the same prediction. 

In this study, each subject in the experimental (low b) group was seated in a 
swivel chair located between the two-light board and a large mirror. Each trial 
began with the subject facing at right angles to the board and to the mirror.’ 
The signal stimulus consisted of two arrows, one of which was lighted to start. 
each trial. In a random sequence, one half of the time the arrow was lighted 
that directed the subject to make his prediction at the board, and the other 
half of the time the arrow was lighted that directed the subject to make his 
prediction at the mirror. There were two “prediction” buttons in front of the 
mirror as well as the usual two in front of the board. In this situation, a | 
subject desiring to predict the illumination of the left light would press the | 
left button when facing the board but the right button when facing the mirror 
since, in the latter case, he would see a “mirror image” of the board and make 
his prediction according to the position of the lights in this image. In addition 
to the cognitive variability involved in such a situation, there was considerable 
kinesthetic variability due to the necessity for him to swivel from the signal 
light position to the board (or mirror), back to the signal light for the start of 
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the next trial, thence to the mirror (or board), and so forth. In the situation, 
a subject could choose the more frequent stimulus event on all trials (adopt a 
pure strategy) while pressing a left button 50 per cent of the time and a right 
button 50 per cent of the time. 
In this experiment” 60 subjects were randomized into two groups, one group 
run in the experimental (mirror) condition and one in the control condition, 
m being set at 0.75, and each subject run individually for 240 trials. Both 
groups were run under No Payoff. The control subjects were run, thus, in the 
classical two-light situation. 
The results of this study provided support for the prediction that the average 
p of the experimental subjects would be higher than the average of the controls. 
_ The average proportion of predictions of the more frequent light by the control 
— subjects was 


pe = 0.74 
"whereas for the experimental group the average was 
E: pe = 0.80 


_ The difference between these means is significant at p < 0.05. 


Summary and Conclusions 


To date, three experiments have been conducted concerning various features 
~ of Model I. 
The first experiment confirmed the general features of the model by demon- 
 strating that the prediction strategies that subjects reach differ according to 
the reinforcement inherent in the situation. Under No Payoff, the asymptotic 
- rate of predicting the more frequent event is lower than it is under Reward. 
~ In turn, the prediction rate under Reward is lower than it is under Risk. 
The second experiment supported the specific mathematical structure of 
~ Model I. On the basis of a value of & derived from the Model with data from 
one group of subjects performing under Risk with = 0.75, a prediction was 
~ reached concerning the strategy a second group of subjects would stabilize at 
: under the Risk condition with = 0.65. The predicted value was p2 = 0.758, 
and this was remarkably close to the observed value, p2 = 0.753. 
‘The third experiment was directed toward validating the psychological 
: meaning of one of the important expressions in the model, Ugo tae expected 
"utility of variability. Subjects were given an opportunity to vary the kines- 
- thetic and cognitive experiences associated with their responses without vary- 
ing the prediction represented by those responses. Under these circumstances, 
4 ? increased, as predicted from Model I. 
Thus all three experiments have supported hypotheses derived from Model 1; 
It should be noted, however, that this is a relatively simple model, with few 
variables. It does not incorporate a method for dealing with—measuring or 
_ controlling—the specific utility of gambling. This term refers oo ee to such 
- satisfactions as are inherent in “playing a game with the inachine, winning 
- ona long shot,” and “successfully predicting the rarer event.” | To. handle this 
‘concept, which may well be important in determining subjects’ prediction 
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strategies, a second model was developed that is more general and “richer” 
than Model I. This approach, Model II, can now be presented. At present 
we are beginning to conduct studies testing Model II, and a description of the 
experiments now in progress is included in the presentation of Model II that 


follows. 


Model II 


Let x = the probability of occurrence of the more frequent event, as in 


Model I; 
p = the proportion of times the subject chooses the more frequent 


event, as in Model I; 
a = the marginal utility of a correct prediction when and only when the 
subject chooses the more frequent event; 
b = the marginal utility of a correct prediction when and only when 
the subject chooses the less frequent event; and 
c¢ = the marginal utility of variability. 
The expected utility of a correct prediction, E,(U,), is 


and the utility of variability, U, , is f(p), as in Model I, and is 
U, = cpl — 9) 
The total expected utility of a particular strategy # is 
U(p) = apm + (1 — p)(1 — x) + cp(1 — 9) 
The strategy p that maximizes U(p) is at 


dU(p) 
rT acai 0 
and is 


ax — (1 —-7m) +c 
2c : 


p = 
Ifa = a/c and if 8 = b/c, then 


Tt may be seen from 2 that when a = 6 = ¢, and thusa = B = 1, Model 
ie ee set a special case, the Estes prediction that = a. It will be shown _ 
ater that the case when a = 8 = 1 is not the only case f hich p = a 
cording to Model II. ‘ ee | 

Model I is richer than Model I in several respects. One is that it makes 
possible experimental studies involving differential payoffs for the more fre- 
quent and less frequent events. In addition Model II may lead to the meas- 
urement of an individual’s specific utility of gambling. 
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The predictions Model II yields concerning a subject’s choice and strategy 
behavior may be generally stated as follows: For any 7, 


(1) p = 1 when 
a> (Lame +1 
iF 7 
f and/or 
ra — 1 
es ee 


F @)1> p> 7when 
z= (1-8 +1. — A — 8+ Or - 1) 
; T 


Tv 
~ and/or 
wa 1g re — (an — 1) 
7 1 = 4 1—r 
x (3) » = m when 
3 tb x21) 
; 7 
: and/or 
-, wa — (Qe — 1) 
j vie Le 5 
‘Ss 
© (4) p< a when 
4 (= we Or = 1) 
3 PA AOR AE A EE 
“a Tv 
: 
~ and/or 
4 ee 
4 he ke ae al 
a: Pe, 2 
The first, second, and fourth predictions—that p will be 1, will be between 1 


_ and 7, or will be less than r—are predictions regarding asymptotes that the 
Estes model is not prepared to make. ‘The third prediction—that p will be at 
_a—is the Estes prediction, and occurs here as a special case (or set) in Model 
ELT: ; 

- In Model II, there are two parameters, a and 8, which must be estimated 
before the value of p can be predicted: 


Seu ee (2) 


p 
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Therefore a study is needed in which two groups of subjects are run under 
different 7’s and one level of reinforcement, Ri. Data are required, that is, 
from a study of this design: 


1 T2 


Ry Pi pe 


With 1, and m2 set, and with p, and p2 observed, according to Model II, 


_ (1-8) + ma + 8) 
fu B, 


_ G-= 8) + mle + 8) 
Pema 2 


Solving these two equations simultaneously, we obtain estimates as follows: 


Be 2(pir2 — pom) eS 


A Ws meet Ip 
4 — 2th — pe) _ 
Yh eer | 


With 8 and & computed, a study could then be conducted using a different 
a than those above, for example 73, and, under the same reinforcement, R;, 
as was used above, a test of Model II would be a test of the prediction. 


_ (1 = 6) + wla+ 6) 
py = mee 


An important difference between Model I and Model II is that Model IT 
contains separate expressions for the utility of correctly predicting the more 
frequent event and the utility of correctly predicting the less frequent event. 
Thus whereas both models predict that will depend on the reinforcement hing- 
ing on correct predictions, only Model II yields a prediction for the situation in ; 
which the reinforcement contingent on correctly predicting the more frequent — 
event differs from the reinforcement contingent on correctly predicting the less 
frequent event. According to this model, by systematically varying these two 


types of reinforcement it should be possible to induce variations in p from — 


zero to one! 
Consider, for example, the Risk situation used above. The subject was con- 


fronted with two lights, one of which (for example the left one) illuminated on - 


75 per cent of the trials (7 = 0.75) and one of which (the right) illuminated on _ 


25 per cent (1 — w = 0.25). If the subject predicted that the left light would 

illuminate and it did, he received 5 cents. If it did not, he forfeited 5 cents. 
On the other hand, if he predicted that the right light would illuminate and it 

did, he received 5 cents, and he forfeited 5 cents if it did not. This reinforce- 

ment situation may be represented schematically as follows: 


poi sis Sela \ ste tes Ok 


a ie ere 


eer | bey 
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left right 

(0.75) (0.25) 
prediction confirmed +5 cents | +5 cents 
prediction disconfirmed | —5 cents =. cents 


Under such a reinforcement situation, it is clearly to the subject’s monetary 

advantage (in terms of expected value) always to predict the illumination of 

the left light, and we presume that the reason subjects fail to follow this strategy 

is that they are attempting to maximize expected utility rather than expected 

value, and that the total utility inherent in the situation includes other com- 

ponents besides the monetary one—the utility of variability, for example. 
Suppose, now, that the reinforcement situation were as follows: 


left right 

(0.75) (0.25) 
prediction confirmed +5 cents | +5 cents 
prediction disconfirmed | —5 cents — $1.00 


Under such a situation, the expected value associated with predicting the il- 

lumination of the left light is much higher, and we would expect the utility of 

money to practically override any competing utilities that induce occasional 

selection of the right light, and that » would be closer to unity in this situation 

than in the preceding one. Such a prediction is yielded by Model IT. 
Consider, on the other hand, this situation: 


left right 
(0.75) (0.25) 


+5 cents +5 cents 


prediction confirmed 


prediction disconfirmed | —5 cents +$1.00 


Under this situation of reinforcement, we would expect p, the proportion of 
times the subject predicts the illumination of the left light, to be far lower than 


under either of the preceding situations, and in fact to be near or at zero. 


The above examples illustrate that by varying one monetary value in the 


reinforcement situation, from +d dollars through 0 to —d dollars, we can in- 


duce variation in the subject’s p from zero to one. Such induced variation in 
p, if observed, would constitute partial verification of Model I. 
With an experimental set-up such as that outlined above, it should be pos- 


_ sible to obtain a measure of a subject’s specific utility of gambling. To obtain 


such a measure, it would be necessary to know the subject’s utility of money, 
and this could be determined by the method developed by Davidson et al.’ The 


experimenter could then determine how large a change in money (in one of the 


cells showing reinforcement, as shown above) is necessary to change the sub- 
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ject’s strategy, p, from some stable value to unity This result, under the 
proper experimental conditions, should yield a measure, in monetary units, 
of the subject’s specific utility of gambling. 


A Decision-Making Theory of Learning 


The principal purpose of the models that have been presented is to formalize 
the notions of decision-making theory and to use them to predict stable-state 
behavior in a binary choice situation. It has been shown that the mathe- 
matical models presented above can predict such asymptotes, and do so for 
varying conditions of reinforcement. 

A shortcoming of these models is that they imply nothing about the course 
of learning. In contradistinction, the Estes model'*-!* and the Bush-Mosteller 
model have the power to describe the entire course of learning in the two- 
choice situation, but they have the shortcoming that they fail to explain the 
effects of varying conditions of reinforcement. 

I have been working with Robert Radlow, a National Science Foundation 
postdoctoral Fellow, on extending the decision-making models so that a more 
general class of behaviors can be predicted. The extension of the decision- 
making models to learning behavior is an attempt at unifying the two ap- 
proaches just mentioned, in that it allows prediction of the entire course of 
learning and of asymptotic choice behavior for different conditions of reinforce- 
ment. 

The essential idea of the decision-making theory of learning is to regard an 
individual’s choice on any given trial as the result of his degree of belief (sub- 
jective probability) that a given event will occur as well as the result of the 
utility associated with the event’s payoff. It is hypothesized that on any given 
trial the individual will choose as if he is attempting to maximize his subjec- 
tively expected utility. 

The entire development of the extended models will be presented elsewhere.” 
For the present, it is sufficient to say that the derivations follow closely on 
those of Models I and II presented earlier, the main difference being that 
subjective probability S, is utilized in place of the objective probability 7 
that a particular event will occur on any given trial. Thus, where in Model I 
the predicted asymptote was 


p=ar—-¥)+ (1) 
it can be shown that for the extended model the prediction equation becomes 
b' = oS, —- 4) + 8 (1a) 


Similarly, the prediction yielded by Model II 


pee ath (2) 
now becomes 
y= CO) +S le+s) o 


For anempirical test of this model, it is necessary to have a theory and a 
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method for the measurement of subjective probability S,. We have devised 
one approach in an experimental test of the prediction 1, above. 

Tt may be seen from 1, that when a = 1.00, S, = p’. Since there is con- 
siderable empirical evidence that a is very close to unity in the No Payoff 
condition, the behavior of subjects under this condition may be taken as a first 
estimate of S,. It may then be assumed that subjective probability is learned 
in a way that is described by an Estesian exponential function,!8 with pi taken 
to be 0.50. 

In other words, the Estesian learning curve is here reinterpreted as being a 
representation of the subjective probability function, and the curve is used to 
obtain S, for prediction equation (1,). 

Is there any justification for calling an Estesian learning curve a “subjective 
probability curve’? In general terms, this redefinition transforms the Estes 
model, an associative model that attaches stimulus “elements” to responses, to 
a new associative model that attaches stimulus elements to beliefs. In brief, 
then, the approach may be interpreted to assert: (1) that simulus elements are 
connected with beliefs that an event Ey will or will not occur, (2) that an 
increase in degree of belief (subjective probability) that E; will occur results 
when the belief is confirmed, and (3) that the relation between degree of belief 
and choice behavior is not a simple one, since choice behavior also reflects what 


_ we may call a strategy to maximize subjectively expected utility. 


Experimental Test of the Learning Model 


In an initial test of the learning model* 40 subjects were randomized into 
two groups of 20 each. Both groups were run in the two-light setup for 200 
trials. For these sessions, r = 0.70, and thus the two lights occurred in the 
ratio of 70:30, randomized into blocks of 20 trials. 

One of the groups of subjects was run under the No Payoff condition. Their 
responses were used to generate an inferred subjective probability curve. 

The other group of 20 subjects was run under conditions of Risk. The 
average course of learning in this group was predicted from the inferred sub- 


_ jective probability curve obtained from the first group’s data and from the 
- learning model. 


S, , taken from each block of the inferred subjective probability curve, was 
used to derive a predicted learning curve of the group under Risk, by applica- 


tion of 


fn Sp, — 78) 72 (1) 


where the value of a was taken from previous research with other subjects 


under these reinforcement conditions,” and was @ = 1.718. Ficure 1 shows 
the predicted values and those observed in this study, and reveals that the 
observed values are in excellent agreement with those predicted from the 


learning model. ee 
It is interesting to note that because the model assumes that subjective prob- 


ability is learned independently of the condition of reinforcement, it is pos- 


sible to obtain data to test the model without having to run a group under 


_ No Payoff conditions. That is, a group could be run under any condition of re- 
inforcement for which & is known, and then the “no payoff subjective prob- 
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ability curve” could be derived by 


/ iW 1 
n 72 
Sait pu? bis 


These values could then be used to derive the predicted learning curve for 
subjects under any other reinforcement conditions. 


1.00 

0.90 

0.80 Posse xe 
0.70 tag 

0.60 4 

iste epee 


0.40 


o——o Observed proportions, 7 = 0.70 


0.30 x---x Predicted proportions obtained 
from equation Iq 


0.20 


0.10 


0.00 


2 Sr a aS Se 8h ee 


FicurE 1. The risk condition: observed proportions and predictions. 


Summary 


From the general hypothesis that humans may be expected to behave as is 
they were attempting to maximize subjectively expected utility, two modelf 
have been developed for the asymptotic choice behavior of individuals in a 
binary choice situation, and a model has been developed to describe the course 
of learning of strategies in this situation. In these models, the frequency with 
which the subject predicts the occurrence of the more probable event is taken 
to represent his strategy, and it is hypothesized that this strategy will vary, 
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dependent on the various utilities inherent in the situation. For this reason, 
it is expected that the person’s strategy will vary as induced reinforcements are 
varied, and this expectation has been confirmed in experimentation. 


So CO NAME wo Nn 


Se Se 
no —& © 


— 
ew 


_ 
on 


Ee Fe Se Ee 
SS 0000) 3 ~ oy 


he Sa) te St 
pereivey Set nS ros RO. pene 


28. 


29. 
30. 


. SIEGEL, S. & D. A. GOLDSTEIN. 


References 


. Bernovtii, D. 1738. Specimen theoriae novae de mensura sortis. Comentarii Aca- 


demiae Scientiarum Imperiales Petropolitanae. 5: 179-192. Translated by L. Sommer, 
1954, in Econometrica. 22: 23-36. 


. Ramsey, F. P. 1931. Truth and probability. Jn The Foundations of Mathematics 


and Other Logical Essays. Harcourt Brace. New York, N. Y. 


. Von NEumAnN, J. & O. MorGENSTERN. 1955. Theory of Games and Economic Behav- 


lor. 2nd ed. Princeton Univ. Press. Princeton, N. J. 


. Arrow, K. J. 1951. Social Choice and Individual Values. Wiley. New York, N. Y. 
. SAvAGE, L. J. 1954. The Foundations of Statistics. Wiley. New York, N. Y. 
. Epwarps, W. 1954. The theory of decision making. Psychol. Bull. 51: 38-417. 
. Davipson, D., P. Supprs & S. SrecEL. 1957. Decision Making: An Experimental 


Approach. Stanford Univ. Press. Stanford, Calif. 


. Luce, R. D. 1959. Individual Choice Behavior: A Theoretical Analysis. Wiley. 


New York, N. Y. 


. SIEGEL, S. 1959. Theoretical models of choice and strategy behavior: Stable state be- 


havior in the two-choice uncertain outcome situation. Psychometrika. 24: 303-316. 


. SIEGEL, S. & L. E. Fouraxer. 1960. Bargaining and Group Decision Making: Ex- 


periments in Bilateral Monopoly. McGraw-Hill. New York, N. Y 


. THRALL, R. M., C. H. Coomps & R. L. Davis (Eds.). 1954. Decision Processes. 


Wiley. New York, N. Y. 


. Topa, M. 1956. Guessing sequences under various conditions of payoff. Japan. Psy- 


chol. Research. 4: 11-23. 


. SIEGEL, S. 1957. Level of aspiration and decision making. Psychol. Rev. 64: 253-262. 
. Estes, W. K. 1950. Toward a statistical theory of learning. Psychol. Rev. 57: 94- 


107. 


. Estes, W. K. 1954. Individual behavior in uncertain outcome situations: An inter- 


pretation in terms of statistical association theory. Im Decision Processes. : 127-137. 
R. M. Thrall, C. H. Coombs & R. L. Davis, Eds. Wiley. New York, N. Y. 


. Estes, W. K. 1959. The statistical approach to learning theory. Jn Psychology: A 


Study of A Science. 2. S. Koch, Ed. McGraw-Hill. New York, N.Y. 


. Estes, W. K. & C.J. Burke. 1953. A theory of stimulus variability in learning. Psy- 


chol. Rev. 60: 276-286. 


. Estes, W. K. & J. H. StravcHan. 1954. Analysis of a verbal conditioning situation 


in terms of statistical learning theory. J. Exptl. Psychol. 47: 225-234. 


. Grant, D. A., H. W. Haxe & J. P. HornsetH. 1951. Acquisition and extinction of a 


verbal conditioned response with differing percentages of reinforcement. J. Exptl. 
Psychol. 42: 1-5. 


. Jarvix, M. E. 1951. Probability learning and a negative recency effect in the serial 


anticipation of alternative symbols. J. Exptl. Psychol. 41: 291-297. 


. Hake, H. W. & R. Hyman. 1953. Perception of the statistical structure of a random 


series of binary symbols. J. Exptl. Psychol. 45: 64-74. 


. Goopnow, J. J. 1955. Determinants of choice distributions in two-choice probability 


situations. Am. J. Psychol. 68: 106-116. 


. Busu, R. R. & F. MostELter. 1955. Stochastic Models for Learning. Wiley. New 


York, N. Y. 


. Messick, S. J. & C. M. Sorrey. 1957. Probability learning in children: Some ex- 


ploratory studies. J. Genet. Psychol. 90: 23-32. 


. Humpsreys, L. G. 1939. Acquisition and extinction of verbal expectation in a situa- 


tion analogous to conditioning. J. Exptl. Psychol. 25: 294-301. : 
1959. Decision-making behavior in a two-choice un- 


certain outcome situation. J. Exptl. Psychol. 57: 37-42. 


. Smecet, S. & R. P. ABELSON. Stable-state two-choice behavior: A test of a decision- 


making model. In preparation. ‘ iets ; 
Sacer s. R. Rasnow uC. M. WeisER. The utility of variability in the two-choice 


situation. In preparation. f ; : 
Raptow, R. & S. Stecet. Decision making and learning. In preparation. 7 
Raptow, R. & S. Srecet. Probability learning and choice behavior under differing 


reinforcement conditions. In preparation. 


SOME CHARACTERISTICS OF CHOICE BEHAVIOR 
IN RISKY SITUATIONS* 


C. H. Coombs and D. G. Pruitt 
University of Michigan, Ann Arbor, Mich. 


It has been observed that, before deciding between alternatives, people often 
enumerate the possible outcomes of choosing each alternative. They give each 
outcome a value representing its desirability, and often form some sort of 
estimate of the probability of occurrence of each outcome. Scientists have long 
believed that it should be possible to predict the direction of decision on the 
basis of these estimates of the value and probability of outcomes. Several 
models for making such predictions have been proposed (for example, Bernoulli, 
1954; Edwards, 1955) based on the concept that people choose the alternative 
with the greatest expectation (the sum over outcomes of the probability times 
the value of each outcome). These models are weak because they can be made 
to predict such a wide range of behavior that it is difficult to design a crucial 
experiment. This weakness has led us to seek new directions for model build- 
ing. 

Two concepts upon which such a revision might be based are suggested in 
the current literature, both concerning elements of risk in alternatives: (1) 
“probability preferences,”’ the concept that people are more attracted to cer- 
tain probabilities of winning or losing than to others (Edwards, 1953); and 
(2) ‘‘variance preferences,” the concept that people prefer certain dispersions 
of values of possible outcomes (Fisher, 1906; Allais, 1953). The present ex- 
periment was designed to find out whether probability and variance preferences 
exist and to discover any interactions that may occur between them. In addi- 
tion, a model of variance preferences was tested that made more specific pre- 
dictions than the expected utility interpretation (Bernoulli, 1954), a version of 
the expected value model. 

An earlier report on this study has already been published (Coombs and 
Pruitt, 1960). In the present paper, the model for preferential choice behavior 
will be reviewed and the earlier report summarized. Then some additional 
analyses will be presented concerning inconsistency and transitivity of prefer- 
ence, group scales, and sex differences in probability and variance preferences. 


A MODEL FOR VARIANCE PREFERENCES 


The model is based upon a theory of preferential choice and the unfolding 
technique discussed in detail elsewhere (Coombs, 1952). Consider a set of 
alternatives, each having two outcomes at probabilities p and q respectively 


and all alternatives having the same expected value. The amount of variance - 


of each alternative may be considered a point on a line or continuum of vari- 
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ance.* Assuming that an individual has an ideal variance, corresponding to 
a point on this same continuum, and that he prefers, of any two variances, the 
one nearer his ideal variance, then his preference ordering of the alternatives 
is the rank order of the alternatives obtained by folding this scale at his ideal 
point: the alternatives being ranked in order of increasing absolute distance 
from his ideal point. It will be convenient to speak of the variance scale gener- 
ically as a joint scale or J scale (both stimulus points and the ideal points of 
individuals are located on it); and the preference ordering of an individual, 
which is a folded J scale, will be called an J scale. 

Consider the midpoints between stimuli on the J scale. The order of the 
midpoints from left to right on the J scale is determined in part by the relative 
distances between the stimuli. For example, if the distance between the first 
two stimuli, A and B, is greater than the distance between the next two stimuli, 
C and D, that is, if AB > CD, then the midpoint of stimuli A and D precedes 
the midpoint of stimuli B and C. The J scale is divided into segments that 
are bounded by the midpoints. To each such segment there corresponds a 
unique ordering of the absolute distances of the stimuli from any point in the 
segment. ‘This ordering is, by hypothesis, the preference ordering of an in- 
dividual whose ideal point lies in that segment. 

It is evident that while the stimuli may be in the same rank order on the J 
_ scale for different individuals, the relative distances between the stimuli may 

not be the same if the measures of the stimuli are subjective rather than ob- 

jective. A consequence of this is that the order of the midpoints may be dif- 
_ ferent for different individuals, and the sets of preference orderings correspond- 
ing to the different orders of midpoints will not be completely identical. It is 
convenient, therefore, to make the following distinction: if the order of the 
midpoints appears to be the same for all individuals, the J scale is called a 
— “quantitative J scale” (that is, at least a common ordered-metric scale), whereas 
_ if the midpoint order is not the same but only the rank order of the stimuli, 
the J scale is called a “qualitative J scale” (that is, at most, a common ordinal 
scale). 

In summary, a preference ordering of an individual, called an J scale, is re- 
- garded as a folded J scale. If a set of preference orderings, obtained from a 
~ number of individuals, may be unfolded to construct a J scale with no contra- 
 dictory order relations on intervals between stimuli, the J scale is called a 
_ quantitative J scale. If the set of J scales may be unfolded to construct a Ey, 
scale on which only the order of the stimuli is common, the J scale is called a 
- qualitative J scale. 

7 METHOD 

The stimuli were two-outcome bets, such as the following: 1/3 chance to win 
50 cents, and 2/3 chance to lose 25 cents. All bets had an expected value of 
- zero. The bets were organized into five sets. Sets I, I, and TH each con- 
g tained 6 bets that differed in variance but had the same probability of winning. 
Inset I, P = 1/3; in set II, P = 1/2; and in set III, P = 2/3. The same six 
Z 


2 Pott te ; : : a — 5). 
5 * In constructing the stimuli of this experiment, variance was defined as V = pq(a - 
_ However, any mats of dispersion is admissible under the model as a definition of variance. 
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variance levels were used in all three sets: $0.12, 0.30, 1.00, 5.00, 25.00, and 
100.00. Sets IV and V each contained 5 bets that differed in probability of 
winning but had the same variance. The variance of set IV was $1.00; of set 
V, $25.00. The five differing probabilities were 1/6, 1/3, 1/2, 2/3 and 5/6. 
Sets I, II, and III were used to study variance preferences under constant — 
probability treatment; sets IV and V were used to study probability preferences 
under constant variance treatment. Within each set the bets were exhaustively 
paired, and each pair was presented 8 times. 

The bets were given to the subjects in dittoed booklets constructed so that 
one replication on all pairs was completed before the next was started. The 
Ss were told to choose, in each pair, the bet they ‘‘would prefer to play” and 
to indicate this decision by making a check in a box beside the bets. Ninety- 
nine students at the University of Michigan served as subjects. 


SUMMARY OF FINDINGS PREVIOUSLY REPORTED 


Consistency and Transitivity of Preference 


An information measure was used to eliminate the few Ss in each set who 
were essentially random in their choices. This measure provided a convenient 
index number to characterize an individual’s degree of inconsistency of choice 
in each set of stimuli. Comparison of these indices across sets of stimuli re- 
vealed that individual differences in inconsistency are statistically significant 
and moderate in magnitude. 

Weak stochastic transitivity was tested and found to be satisfied for most 
individuals. When this condition is satisfied, the bets can be simply ordered 
from left to right for each individual so that each bet is preferred at least one 
half the time to any bet to the right of it. This is taken to be his preference 
ordering or J scale. 


Unfolding Analysis of Preferences 


Variance preferences. The preference orderings (J scales) for each set of 
stimuli were unfolded, and a best quantitative J scale was constructed. In all 
three sets (I, II, and IIT), the vast majority of preference orderings fit the same 
quantitative J scale, and most of those that did not fit deviated in minor ways, / 
apparently as a result of inconsistency. Thus the results gave strong support — 
to the proposed model of variance preferences. However, it was not possible 
to answer with assurance the question whether a common quantitative or only 
a common qualitative J scale of variance underlies choice. Although the 
findings seem to support the former conclusion, this might have resulted from 
the choice of stimuli. 

Probability preferences. Again the preference orderings were unfolded for — 
each set. However, in this case, only about 60 per cent of the cases in each 
set fitted the same quantitative J scale, a scale of increasing probability of 
winning. In most of the preference orderings that did not fit, the trouble 
seemed to lie with bets at 1/2 chance of winning, which were too much preferred. 
When these bets were eliminated from the analysis, a much larger proportion 


of the cases fitted the same quantitative J scale of increasing probability of 
winning. 


’ 
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Stability and Interaction of Preference Patterns 


For almost every individual, the probability preference patterns at the two 
levels of variance (sets IV and V) were remarkably similar, suggesting that 
probability preferences are stable in the face of changing variance. On the 
other hand, an individual’s variance preferences were often quite different at 
different probability treatments (sets I, II, and III). These differences were 
largely a function of probability preferences: at probability levels that people 


preferred (as shown by their behavior in sets IV and V) they preferred more 
variance. 


TABLE 1 
DistRIBUTION oF H Wiruin Stimutus Sets 


Stimulus sets* 
Ht Variance preferences j Probability preferences 
I II lil IV V 
0-0. 1000 15 eH 21 6 7 
0.1001-0.2000 14 15 14 14 13 
0.2001—0.3000 14 16 9 18 7 
0.3001—0.4000 14 12 13 7 18 
0.4001-0. 5000 13 9 10 12 11 
0.5001—0.6000 10 3 10 20 13 
0.6001-0.7000 12 3 10 6 15 
0.7001-0.8000 i) 1 5 8 10 
0.8001—0.9000 ; : : ‘ 
0.9001—1 .000 
Hil 0.368 0.229 0.367 0.434 0.443 


* The entries in all but the bottom row refer to number of Ss. 
{ Higher H indicates greater inconsistency. 
t His a mean H. 


ADDITIONAL ANALYSES 


Inconsistency of Preference 


The obtained distribution of the information measure of inconsistency, H, is 
given in TABLE 1. The average degree of inconsistency for each of the stimulus 


sets is contained in the bottom row of the table, indicating that probability 
_ preferences were most inconsistent, variance preferences at P’s of 1/3 and 2/3 


next most inconsistent, and variance preferences at 1/2 most consistent. 

In a previous study of inconsistency of preferences (Coombs, 1958) it was 
shown that laterality of stimuli on the J scale relative to an individual s ideal 
affected the order of magnitude of his inconsistency. For individuals toward 


the extreme of a J scale the pairs of stimuli are predominantly unilateral, and 
for individuals with intermediate ideals, there are more bilateral pairs. Vari- 
E ability in the location of an individual’s ideal inflates the inconsistency of bi- 


lateral judgments but has no effect on unilateral judgments. Since all individ- 


uals are assumed to have variable ideals, one would expect the inconsistency 


measure of an individual’s preference to be higher the nearer he is to the middle 
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of the J scale. These conclusions are borne out by the results presented in 
TABLE 2. Stimulus sets I, II, and III have 6 stimuli each, designated A - 
to F in order of increasing variance, in this table. Sets IV and V have only 
5 stimuli, and the symbols A to E correspond to the probabilities 1/ 6 to 5/6 
respectively. It is evident that, for each set of stimuli, inconsistency is great- 
est for individuals centrally located on the J scale and diminishes toward either 
end. 


TABLE 2 
RELATION OF INCONSISTENCY TO IDEAL 


Stimulus sets 


I Il Ill Iv V 
H* [Md HiINt| H |MadH|N| H |MdH|N| H |MdH|N| H |MdH|N 
Most 

preferred 

stimulus 
A 0.2157} 0.1630) 34) 0.1802) 0.1450} 18} 0.2582} 0.1948] 16} 0.4070) 0.3442) 33) 0.4566) 0.4895) 31 
B 0.4045} 0.3959! 4) 0.2167) 0.2189) 4) 0.3500} 0.3841) 13) 0.6252) 0.5807) 3) 0.6884) 0.7401) 4 
Cc 0.4520] 0.4979] 16] 0.2846] 0.2657| 11} 0.5689) 0.5743) 8) 0.5231) 0.5173} 21] 0.5366] 0.5295) 15 
D 0.4884! 0.4824) 12] 0.4138] 0.4273} 12) 0.5188} 0.5452| 14) 0.4288) 0.4341) 9) 0.4512) 0.4053) 6 
E 0.4586} 0.4471) 9) 0.2547) 0.2627) 15) 0.3626) 0.3472) 5) 0.2978) 0.3253) 29) 0.3341) 0.3397| 38 
F 0.2112) 0.2169) 17} 0.1095) 0.0362) 36) 0.2359) 0.1812) 37 


* H is the mean H. 
1 H indicates inconsistency. 
t N indicates number of Ss, 


Strong Stochastic Transitivity 


Whether inconsistent preferential choices satisfy strong stochastic transi- 


tivity is of some interest. Strong stochastic transitivity (s.s.t.) is defined as 
follows for any triple (a, b, c): - 


P(a,b) > 0.5 and P(b,c) > 0.5 <=> P(a,c) > max {P(a,b), P(b,c)} 


It is clear that data satisfying this condition would also satisfy the condition ! ; 
for weak stochastic transitivity. The distinction is made clear in this example. _ 
Suppose that P(a,b) = 0.60 and P(b,c) = 0.70. If P(a,c) is at least 0.50 then 
weak stochastic transitivity is satisfied by the ordering abc. Strong stochastic 
transitivity requires that P(a,c) be at least 0.70. It is obvious that a monotone 
transformation of inconsistency into psychological distance, as in the Thurstone 
(1927) and Luce (1959) models, requires strong stochastic transitivity. 

While no simple statistical test exists for testing whether the data of this — 
experiment satisfy strong stochastic transitivity, a simple tabulation of the 
degree to which they do is of interest. This tabulation is presented in TABLE 
3. Strong stochastic transitivity is satisfied by only a small number of those 
subjects whose preference orderings satisfy weak stochastic transitivity (column 
b). It must be kept in mind that each pairwise probability is based on only 
eight observations and it is to be expected that even if strong stochastic transi-_ 


: 
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tivity held in principle, a certain number of Ss would fail to satisfy it. Un- 

fortunately no statistical test may readily be made of whether the observed 

eee is a significant departure: one only forms the impression that this may 
€ so. 

Further evidence is contained in the number of triples that fail to satisfy 
strong stochastic transitivity. In stimulus sets I, II, and III, there are 20 
triples imbedded in the weak stochastic ordering of the 6 stimuli in each set. 
In sets IV and V, with 5 stimuli, there are 10 triples in each. The failure of 
an individual’s ordering to satisfy strong stochastic transitivity is due to the 
failure of one or more of these triples. Column c of TABLE 3 contains the total 
number of triples, in each stimulus set, multiplied by the number of Ss. Col- 
umn d gives the number of such triples that fail to satisfy strong stochastic 
transitivity and column e indicates the proportion. It is apparent that, on 
the average, about 25 per cent of all triples fail to satisfy strong stochastic 


TABLE 3 
NUMBER OF Ss AND TRIPLES SATISFYING STRONG STOCHASTIC TRANSITIVITY 


Stimulus set a b ¢ d e€ jj 
ig 86 9 1720 442 0.26 5.74 
II 91 31 1820 308 0.17 Bye il) 
III 85 14 1700 453 OF27 6.38 
IV 89 18 890 219 0.25 3.08 
V 88 13 880 243 0.28 3.24 


(a) Total number of Ss satisfying weak stochastic transitivity. 

(6) Total number of Ss satisfying strong stochastic transitivity. 

(c) Total number of triples in the weak stochastic orderings. 

(d) Total number of triples failing to satisfy strong stochastic transitivity. 

(e) Proportion of triples failing to satisfy strong stochastic transitivity. 

(f) Mean number of triples failing to satisfy strong stochastic transitivity for those Ss 
who depart from strong stochastic transitivity. 


- transitivity. For those Ss who fail to satisfy strong stochastic transitivity, 
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the average number of triples in their rank order that violate this condition 


- is given in the last column of the table. This indicates that failure to satisfy 
_ strong stochastic transitivity is not a consequence of just one or two triples 


but is perhaps a significant departure. 

Considerations of the laterality of stimuli on the J scale relative to an in- 
-dividual’s ideal (Coombs, 1958) also suggest that strong stochastic transitivity 
would be less often satisfied by people whose ideals are nearer the middle of 
the distribution. Rather than present all the results in detail a summary is 
contained in TABLE 4. All variance preference J scales are divided into three 


_ groups, those beginning with one of the end stimuli (A or F) and those begin- 


ning with an intermediate stimulus. The number of such preference scales 
and the number of triples in their preference scales that satisfy strong stochas- 


tic transitivity are presented in TABLE 4. As is evident, the expectation is 
‘borne out. The results on probability preferences show the same trend, but 


not as strongly, probably due to the fact that there are fewer stimuli (and hence 
less effect of laterality), and because of the strong preference for P = 1/2. 
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Group Preferences 


Preference scales based on the pooled data for all individuals in a stimulus 
set are of some interest. A group preference scale may be constructed by ma- 
jority choice on each pair if weak stochastic transitivity holds for the group, 
with each individual getting one vote on each pair. The result is a rank order 
that is a maximum likelihood preference scale for the entire group. The re- 
sults are presented in TABLE 5. 

Each of these group scales may be seen to correspond to an J scale and the 
corresponding J scale number may be obtained by counting the number of 


TABLE 4 


NuMBER OF SUBJECTS AND TRIPLES SATISFYING STRONG STOCHASTIC 
TRANSITIVITY RELATED TO IDEAL 


a b c d e hi 
A 64 14 0.219 1280 1035 0.809 
BtoE 106 i 0.066 2120 1513 0.714 
F 83 32 0.386 1660 1361 0.820 


(a) Number of J scales. 
(b) Number of Ss satisfying strong stochastic transitivity. 


(c) Proportion of Ss satisfying strong stochastic transitivity. 
(d) Total number of triples. 


(e) Number of triples satisfying strong stochastic transitivity. 
(f) Proportion of triples satisfying strong stochastic transitivity. 


TABLE 5 
GrouP PREFERENCE SCALES 


Stochastically-dominant group Corresponding J scale 
preference scale number 

Probability treatment 

P = 1/3 CBADEF 4 

P= 1/2 EDCBAF 11 

P = 2/3 DECBAF 10 
Variance treatment 

Low 32415 5 

High 34521 8 


midpoints crossed to obtain the group scale and adding one. These numbers 
are given in the last column of the table. It is evident that, as a group, maxi- 
mum variance is preferred at 50-50 odds and more variance is preferred at 
favorable than at unfavorable odds. At higher variance the group tends to 
prefer odds more in their favor, although the prior analysis of variance (Coombs ~ 
and Pruitt, 1960) suggests that this is not a significant difference. In all cases 
the group scales are folded J scales and are different under different treatments. 

Another way of characterizing the preferences of a group is through the dis- 
tribution of the ideals (first choices) of its members. Such distributions:are 
shown in the histograms of FicuREs 1 and 2. In all cases it will be seen that 
the distributions are U-shaped, with roughly one third of the subjects favoring 
each of the two extreme stimuli and one third favoring the intermediate stimuli. 
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A method frequently used for obtaining a group scale is to determine the 
total number of times each stimulus has been preferred in its pairwise presen- 
tation with all others. This vote-count method for preferential choice has 
been shown to be not independent of other alternatives (Coombs, 1958) when 
preferences are generated from a joint scale, and it has not been used here. 
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Ficure 1. Frequency of first choices in variance preferences. 
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Sex Differences 


Inconsistency. In TABLE 6 is shown the mean H for men and women sepa- 
rately on variance and probability preferences, and TABLE 7 contains an anal- 
ysis of variance. It is clear that the men are more consistent than the women 
in their preference behavior in the case of both variance and probability pref- 
erences. There is no significant interaction between sex and treatments in 
the case of either variance or probability preferences. 

Transitivily. These differences are not worth reporting in detail. There 
are slightly more intransitive I scales, and J scales with ties, for women than 
for men, but these may be attributed to the greater inconsistency of the women. 

Preference. A distribution of the first choices of men and women in their 
variance preferences reveals a significant tendency for women to fold the J 


TABLE 6 
MEAN H FoR VARIANCE AND PROBABILITY PREFERENCES OF MEN AND WOMEN 


Stimulus sets 


Sex 
i Il III IV V 
Male 0.3203 0.1902 0.3114 0.3834 0.3803 
Female 0.4025 0.2570 0.4083 0.4708 0.4893 
TABLE 7 
Test oF SEx DIFFERENCES IN INCONSISTENCY OF VARIANCE AND PROBABILITY PREFERENCES 
Variance preferences Probability preferences 
Source 
df MS F df MS F 
Sexes 1 0.4872 4.091* 1 0.4561 4.925* 
Error 97 0.1191 97 0.0926 


Note: interaction of sexes and probability treatments not significant by inspection. 
* Significant beyond the 0.05 level. 


scale of variance, whereas men are more evenly distributed. This is borne out 
in the 3 fourfold tables presented in TABLE 8. 

The probability preferences also present a trend in sex differences, and the 
relevant data are contained in TABLE 9. Women tend to prefer “fair” bets 
and positively-skewed bets more than men, whereas the men seem to prefer 


the negatively-skewed bets more: these are the “almost sure things” with small 
probability of a large loss. 


CONCLUSIONS 


Findings on variance preferences give strong support to a model which as- 


serts that, at each probability of winning, the individual has an ideal level of . 
variance and will choose, of any two levels of variance, that which is closer to 


his ideal. A similar model for probability (skewness) preferences appears to 
hold, except that 50-50 bets are excessively preferred. 
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A vast proportion of the Ss were significantly consistent in their preferences 
and exhibited weak stochastic transitivity. On the other hand, strong sto- 
chastic transitivity was not met in most cases. Inconsistency of preferences 
and failure to satisfy strong stochastic transitivity were observed to be greater 
for individuals whose ideals are intermediate on the J scale, compared with 
those whose ideals are extreme. This is to be expected from the effect of lat- 
erality on the inconsistency of preferences. 


y TABLE 8 
Sex DIFFERENCES IN VARIANCE PREFERENCES 


Stimulus sets 
Ki II Tit 
Male Female Male Female Male Female 
Var. prefs. 

Extreme 27 24. 28 25 26 28 
Folded as 28 13 30 13 26 
= ‘Test x? = 4.125 x? = 5.011 n.s 
z tap. 2 <= 0805 df P< 10.05 
e TABLE 9 
= SEX DIFFERENCES IN PROBABILITY PREFERENCES 

Stimulus sets 
4 
IV: Low variance V: High variance 
Male Female Male Female 
1/6 and 1/3 16 20 13 De 
= 1/2 4 17 2 13 
2/3 and 5/6 22 16 26 18 
Test x= 6.71, 2 df x2 = 8.51, 2df 
P < 0.05 PEO), 02 


Group scales based on majority choice were folded J scales, and they revealed 
that for the group as a whole, maximum variance was preferred at odds of one 
half, and more variance was preferred at favorable than at unfavorable odds; 
at the higher variance level the group tended to prefer odds in their favor. 
Distributions of first-choice were U-shaped for all sets of stimuli. 

Women differed from men in that they (1) were more inconsistent, (2) were 
correspondingly less often transitive, (3) preferred intermediate variance more 
often than men, and (4) preferred even money and long shot bets more than 
men. It is not clear whether women’s greater inconsistency is due to the lat- 
- erality effect, in that their ideals tend to be intermediate, or whether they may 
- be said to be intrinsically more inconsistent in this kind of behavior. 
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Part III. Empirical Research Evidence Regarding Decision Making 


STOCHASTIC LEARNING IN RATS WITH 
HYPOTHALAMIC IMPLANTS 


Merrill M. Flood 
Mental Health Research Institute, The University of M ichigan, Ann Arbor, Mich. 


Introduction 


Several investigators have conducted learning experiments with humans 
and with animals designed to test the validity of various stochastic* models 
for learning. Estes! pioneered in this field. Independently, but at about the 
same time, very similar theoretical approaches were being developed by Bush 
and Mosteller,? by Miller and McGill,‘ and by Bales et al.5 

The outstanding work of Bush and Mosteller, especially on the important 
problems of statistical estimation with applications to experimental data of 
various workers, was reported fully in their book on Stochastic Models for 


_ Learning. Recent work by Burke, Bush, Estes, Mosteller, and others is re- 


ported in a volume entitled Studies in Mathematical Learning Theory, edited by 
Bush and Estes.’ Especially noteworthy, more recently, is the theoretical 
approach of Luce,’ as reported in his book, Individual Choice Behavior, and 
in the work of Suppes? and others at Stanford University, Stanford, Calif. 

The present paper reports on intermittent work, started in 1951, with one 
particular stochastic learning model that I have called the “symmetrical 
model.” The characteristics of this mathematical model that are essential for 
the present paper are summarized in the next section. The symmetrical model 
has been compared with some other stochastic models in an earlier paper of 
mine? in which a comparison was made particularly with the combining-of-classes 
model of Bush and Mosteller. Critical aspects of experimental and theoretical 
procedure and meanings are discussed there also. Suffice it to say that, for 
present purposes, the symmetrical model seems worth checking experimentally 
with respect to its range of validity. I have called this range of validity the 
“scope” of the theory. 

Although the work is discussed here in the context of individual learning in 
rats, my interest in it has been and is primarily with reference to organization 
theory. A series of earlier papers” deal with the use of stochastic models of 
this kind as models for decision-making groups of interacting individuals. No 
attempt is made in the present paper to indicate further the use of such sto- 
chastic models in organization science, so that it is best read as a straight- 
forward paper on individual learning. 

The motivation for using rats, rather than human subjects, is threefold: 

(1) Close control, and use of extremes of reward and punishment is possible 


with rats. 
* Estes! referred to his model as “statistical” or “probabilistic,” and Bush and Mosteller? 


to theirs as “statistical” or “mathematical.” I have called’ the approach “stochastic learn- 
ing theory,” because the mathematical models used are ordinary stochastic processes; “sto- 


_ chastic learning” is now a term quite commonly used. 
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(2) Implanted electrodes in the brain of the rat can be used to provide 
stimuli that are either rewarding or punishing, and readings from electrodes 
can provide useful information regarding the rat’s behavior. 


(3) Experimental costs and difficulties are less with rats than with human © 


subjects when they are utilized for extensive observation over a considerable 
period of time. 

The experiments reported here were all conducted with rats having chronic 
implants in the hypothalamic region, so that the rat was rewarded by electrical 
stimulation at the inner tip of the electrode. The rat made his choices by 
pressing pedals arranged rather symmetrically within a cage, and stimulation 
followed immediately upon pressing a pedal if the experimenter’s pattern of 
rewards so required. Details of the apparatus and experimental techniques 
will be presented in a separate paper; a summary is included in A ppendix A. 

The prime objective of the present experimentation is to determine whether 
or not a procedure can be devised for measuring the six behavioral parameters 
of the symmetrical stochastic learning model in a reliable manner for an indi- 
vidual rat. A later objective, granted success with the present objective, is the 
determination of rough limits on the scope of the theory. For example: 

(1) Do the measured behavioral parameters remain reasonably constant for 
an individual rat with changes in such experimental variables as number of 
pedals available, strength of electrical stimulation, location of implant, type 
of reward (such as food versus implant), alteration by surgery or drugs, and 
age? 

(2) Are the measured behavioral parameters adequate to pass predictively 
from free pace to forced pace conditions, from free pace to latency, and so on? 

(3) Do the behavioral parameters vary from rat to rat, and are the parame- 
ters related to other psychophysical measures? 

Concurrently with the rat experiments, there is parallel experimentatior 
with human subjects utilizing monetary rewards rather than chronic implants, 
these experiments will be reported elsewhere. 


The present paper reports on the analytical techniques used to estimate — 


behavioral parameters, and discusses some preliminary experimental results 
obtained with two rats. It is concluded by my colleagues and me that the 
behavioral parameters can be estimated reliably. No attempt has yet been 
made to determine the range of validity of these estimates, or the scope of the 
theory, as the experimental conditions are changed. 


Symmetrical Model 


The symmetrical stochastic learning model is described by the following 
mathematical relations. We start with a more general stochastic model, 


pt + 1) = M™*'p(), (1) 


where p(/) is the-m-dimensional stochastic vector whose rth component is a 


—_— 


probability ,(/), and where M’* is a square stochastic matrix of order m whose ~ 


8 


elements M{; depend only upon and s. It is supposed that the subject makes 
a sequence of responses among a fixed finite set of alternatives, and that there is 
a probability p,(/), at the end of moment ¢, that response r will occur before the 
end of moment (¢ + 1). The notation is such that 7; denotes the response 
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actually made after moment /, and s, denotes the outcome (stimulus) following 
response 7, . 
The symmetrical model is the special case of 1, in which the following con- 
- ditions are satisfied :* 


Pemera + c'5,01 — 2) 2 OO 5 a, 
© FP = 8a. + wd — 0 A — 8) = CA = 8) 
forr = 2, pare PID ES = 1-2, .o on, 
where 

1/(m — 2) ifm > 2 

b b}——4 

0 igt=— 2: 
and where 

ONae gra 1 

OSs OSs 

(3) 
US ee | 


~ 


Obs CGA: 
_ The quantities a’, b*, andc’ are the three behavioral parameters corresponding to 


outcome s; thus, there are 3m behavioral parameters in all, since there are n 


outcomes possible. 
It is easily seen that 1 and 2 are equivalent to the following conditions: 


Be + 1) = A%p,,() + Bs, * 
Pt + 1) = C*p,,0) + Dp, + EY forr ¥r., 
where 
A=a — 8, 
B= 0’, 
=O 1 =pb* = & 
aie € mk Dy 75). (5) 
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AS | ihe ae 
= (15754). 


In other words, the probability of a response 7, being repeated is a constant 
_ Bs, plus its previous probability multiplied by a constant A*; and the prob- 
_ ability of a new response r is a constant *, plus its previous probability mul- 
- tiplied by a constant D*, plus the previous probability of the previous response 
_r, multiplied by a constant C’. 

* §,,, is the familiar Kronecker delta, such that: 6. = lifx = y, and 5,, = Oif* ¥ y. 


798 Annals New York Academy of Sciences 


It has been shown? that the symmetrical learning model satisfies the com- 
bining-of-classes condition of Bush and Mosteller if and only if the behavioral 
parameters satisfy the following additional conditions: 

C=! 0, for's = "172, <*>, m. (6) 
In this special case, the equations 4 reduce to the following: 
pri(t + 1) = (a* — 5°) pr, (t) + 5, 
prt + 1) = (at — 6*)p,(t) + (1 — a*)/(m — 1), forr A re. 
However, there are other stochastic learning models of the form 1 that are not 


symmetrical but do nevertheless satisfy the combining-of-classes condition;? 
these models are not considered in the present paper. 


Parameter Estimation 


Experimental data always consist of observations on r; and s;, for t = 1, 
2,---,N. The possible values of r; are r, = 1, 2, --- , m; the possible values 
of s; are s; = 1, 2, --- , m; and N denotes the number of successive responses 
in one experimental trial. Of course, the trials may be repeated, with V 
different in each case. 

In addition to the behavioral parameters a‘, b*, and c*, there are also the m 
parameters #(1), which we shall refer to as the “‘state parameters.” The state 
parameters describe the subject at the beginning of any experimental trial. It 
is equally proper to speak of the “state of the subject at moment ?” as the 
stochastic vector p(t); however, for convenience, we shall use state parameters 
to indicate the starting state for a particular experimental trial. 

Maximum-likelihood estimates would be adequate, for our purposes, but are 
often difficult to obtain. For data r,; and s; from a single experimental trial, 
these maximum-likelihood estimates would be defined by the following rela- 
tions. First, we define quantities f(t) recursively as follows: 


Afi +1) = M(t) for #¢= 1)2)--- OWN, (7) 


where M "* and p(1) are defined to be the values of the parameters Mj} and — 


p.(1) that maximize the “likelihood” 
N 
LMi, pe(1)] = IT p.(0), (8) 


where the estimated parameters M$ and (1) are required also to satisfy the 
restrictions of the symmetrical learning model. A numerical example is given 
in the next section to illustrate this estimation procedure. 

If several trials are available for analysis, perhaps with different starting 
states and lengths, then the likelihood estimates would be the values of M iG 
and p,(1) that maximize the likelihood 


a. ONG 
Liss pe (1) = TT TT pa), h=1,2,-++,q; &=1,2,-:+,m; ,(9) 


— 


where q is the number of separate trials and N; is the length of trial 7. In this — 


case, of course, the quantities Phirlt) are defined as follows, analogous to the 
definition in 7: 


p(t = 1) = m""5'(t) for (t = A; 25 oni ag l= 1, 2; Ya q). (10) 
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In many instances, the experimenter will determine the sequences of out- 
comes of s,' in advance of the trials. If these sequences are chosen suitably, 
the calculations required to obtain usable estimates of the parameters can be 
reduced greatly. Furthermore, the state parameters may sometimes be 
determined by the nature of the experiment so that they need not be estimated 
from the experimental data. We now consider the estimation problem for one 
design of experiment that takes advantage of such simplification. This method 
is illustrated by a numerical example in the next section, but is not used ex- 
tensively in the present paper. 

If the experiment is designed to require repeated trials, each with the same 
sequence of outcomes and each with the starting state parameters all equal, 
then the parameters may be estimated rather easily. Specifically, for con- 
venience, we start by considering the following design: 


Ni = No=::: = N, = 3, 
pill) = p(1) = ---= put = (1/m), for (= 1,2,-++,9), (At) 
at tar (p= 1, 2) + <='5 gi pel, 2.3). 
Furthermore, for convenience, we start by considering the case in which 
i —"3, n aa (12) 


_ Thus, the subject chooses one from among three possible choices for a sequence 
of three responses in each of q trials, and the outcome is always the same from 
among two possible outcomes (for example, outcome No. 1). As a further 
matter of notation, since we shall presently be concerned only with the parame- 


ters a, 6, cl, from the matrices M", we shall temporarily drop the superscripts 


corresponding to s and have simply: 


a b ate: 
il igo C oie ae 
M = Wi 
ee ee is : 
2 
C a 1—6-c¢ 
Mw = b a b (13) 
f23-- c 
1—a 
C 1—b-—c¢ 5 
Oe a . 
TAPS teres. Ga C 
ieee eset a | 
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There are actually 3? = 27 possible sequences of responses that the subject 
can make in one trial with three responses, each from among three possible 
alternatives represented by all the possible arrangements of the integers 1, 2, 
and 3. However, because of the symmetry of the stochastic model in use, it is 
easily seen that there are only five types of sequences that require separate — 
consideration in the analysis; this is so because the first response in a trial may 
be arbitrarily labeled 1, the first subsequent response that is different (if any) 
may be arbitrarily labeled 2, and the remaining distinct response (if any) may 
be labeled 3. Hence, the five distinctive sequences of responses are: 


111 (includes 222 and 333), 

112 (includes 113, 221, 223, 331, and 332), 
121 (includes 131, 212, 232, 313, and 323), 
122 (includes 133, 211, 233, 311, and 322), 
123 (includes 132, 213, 231, 312, and 321). 


The experimental data may therefore be completely described by five fre- 
quencies, as follows: 


F\,, = the number of times the first response is twice repeated, 

Fy. = the number of times the first response is once repeated, 

F 2, = the number of times the second response differs from the first response 
when the third response is the same as the first response, and 

Fx; and Fj93 are defined similarly. 


I 


Of course, 
Fin + Fue + Pia + Fin + Firs = q- 


With this notation, by substitution in 9, we can obtain the likelihood function 
for this particular design of experiment when it yields observations F;;,. We 
shall now derive such an expression for Lia, 6, cl, after some preliminary calcu- 
lations. Consider first the case Fin. The theoretical probability of pin of 
observing response patterns 111, 222, or 333, is as follows: 


Pin = Pri t+ Poe + P333 = 3Pin, 


where P;jx denotes the theoretical probability of observing the single sequence . 
(i,j,k). Also, since p,'(1) = (14), for (J = 1, 2, --- , g;& = 1, 2, 3), we shall 
temporarily omit the superscript on p!(1). 

We adopt the customary notation that a prime on a matrix denotes the trans- 
posed matrix. Since p(¢) and p(1) are matrices with one column, or equiva- 
lently column vectors, then p’(/) and p’(1) denote their transposition; thus, 
p’(1) = (4, 4%, 44) in our present discussion. It will also be convenient to let — 
én denote the unit column vector with m rows, having unity in its A" row and 
ZeTOS elsewhere; and to let J denote the column vector with m rows and all 
entries unity. Thus, also, (1) = J/3 and p’(1) = J’(3). 

The theoretical probability of response i at moment ¢ is therefore e;'p(t). 
Consequently, 


Prise = [ep )Iles’M'p() Ilex’ M'M‘p(1)] = [p.(1)][p5(2)][oe(3)]. (14) 
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_ A straightforward calculation shows that: 
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We now proceed to calculate Pin, Pu, Pin, Pie, Piss , that correspond to 
the observational data F;;,. It follows easily that: 


a+ 2b pi(2) w 
M*p(1) = 14] (3 — a — 26)/2] = | p2(2) whl © X16) 
(== a—"25)/2 ps(2) (1 —-w)/2 


where fi(2) = w denotes the theoretical probability of response 1 at moment 2. 


_ Similarly: 


2a? — 4b? + 2ab + 6b | 
M'M'p(i) = || -—@ + 282 — ad — 304+ 3 
i —a@ + 23? — ab — 364+ 3 


3w* — 3bw + b pil3) 
= || (—3w* + 3bw — 6 + 1)/2|| = | po(3) ||, (16) 
(—3w? + 3bw — b+ 1)/2 p3(3) 
a? + 46?-+ 4ab + 6ac + 12bc — 6a — 126 — 664+ 9 
M?M'p(1) = Yo —2a? + 4b? — 2ab + 6a + 66 
a — 8b? — 2ab — 6ac — 12b¢c + 66 + 6c + 3 
(17) 


ease st 2a 3) be} pi(3) 
Be ont = 201 — au) (t £3) = "1 pu3) 
1-4- 3w? + 2(1 — 3w)(b + ©) ps(3) 


fu = 3Pin = 3[4][wl[3u? — 3ow + 0] = 3w? — 3bw? + bw. (18) 
Similarly: 
pu = wl3w* + (1 — 3w)él, 
pur = wi{[(1 — 3w") — (1 — 3w)dl, 


pa = ‘ = 4 p see cS Oe |, 


Pin = (; ts ) [1 — 3w” — (1 — 3w)(1+)], 


(19) 


2 


mt as) pe + (1 = 30) +0). 


| Here, for example, we have used the fact that pi2r = 6Pin since the six equally 
likely actual sequences (121, 131, 212, 232, 315, 323) are all included under pi. . 


a 
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The logarithm of the likelihood function 9 is, in this special case: 


od (8) am (atm (2) 
eis ff 12310 =) 


gfiuln pin + fale pus + fron pin + firpi2e + SirsPi28 
— ln6 +e fiuln2!] ? 


In( La, 6, c]) 


where fijx = (Fijx)/q are the relative frequencies. The parameter estimates 
that we seek, therefore, are the quantities @, 6, ¢ that maximize Jn L but are 
subject to the restrictions of 3. 

Useful parameter estimates might be found by solving the following system 
of equations and inequalities, if such a solution exists: 


fu =fin, pu2= fire, pin = fin, pie = fie, pies = firs, 


20 
O<e<1, 045 61) “Ose Sayy AO ae a 


Of course, the system of relations 20 should have a solution if the type of 
stochastic model used is the one yielding the observations and if the number of 
observations is great enough. When the system 20 does not have a solution, 
because of the inequalities, it can be used to obtain initial values to be used in 
an iterative procedure for determining actual likelihood estimates. 

It is quite easy to find the explicit solution of 20, if it exists, as follows: 


w = fin + fir, 

me 2 (1 + 38°) (1 — #) 

Cay is ie oo) ieee: — fun |, a 
Fee 2 (1 — 3@°)(1 — @) 

2% i mg aa = oe ~ fu, 

a = 3@ — 26. 


After @, b, and ¢ are computed according to 21, it is only necessary to check 
that the values found satisfy the other requirements of 20 in order to ensure 
that they constitute the desired solution; this means checking the inequalities 
and the equations for fi and fis . 

Exactly the same formulas apply in any case when a, b, c are abbreviations — 
for a’, b°, c*, whatever the value of s, provided only that s remains constant 
throughout the set of trials analyzed. Accordingly, independent estimates of 
a‘, b*, and c* may be obtained by selecting from among all the trials just those 
for which s has the desired value for the first two responses. 

Formulas exactly analogous to 21 can also be found, in a straight-forward 
manner, when there are more than three responses in the replicated trials. 
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For example, in the case of four responses per trial, the observational data 
would be the following set of relative frequencies: 


fun, fur, fia, fire , furs, frm, fiw, firs, fren, fire , 


fires , fresi » firs , Sirss - 


_ The 14 theoretical probabilities p;;.:, that correspond to these 14 observed 
relative frequencies, are each a function of the parameters a, b, c. Some sub- 
set of the 14 equations ;;.1 = fi;.1 could be used to determine estimates of a, 
b, and c; again, it would be necessary to check that these estimates satisfy the 
requirements of 20. Alternatively, the likelihood function could be used to 
obtain estimates in this case. It is already quite clear that the algebraic and 
_ computational complexities mount very rapidly as the number of responses is 
- increased beyond 3 or 4 per trial. 

_ Formulas analogous to 21 can also be found, in a straightforward but tedious 
_ manner, if the value of s, is not constant over the set of responses but the 
sequential pattern of s; is the same for each trial. For example, if the values 

of s; are (1, 2, 1), then the case of four responses per trial would yield 14 equa- 

_ tions between #;jx2 and f;;x2 from which the six parameters could be estimated. 

_ On the other hand, if the values of s; were (1, 1, 2), then the first three responses 

_ would yield estimates of the parameters corresponding to s = 1, and the four- 

_ response data could be used to estimate the remaining three parameters cor- 

" responding to s = 2. As always, maximum-likelihood estimates represent a 

4 superior alternative procedure when the calculations are feasible. 

a 


<a 


> eS 


A fundamentally different method of estimation, whether or not maximum- 
- likelihood techniques are used, would be to make estimates disregarding the 
inequalities in 20 and then “round off” as necessary to satisfy them. For ex- 
ample, if an estimate of —1.5 were obtained for some parameter, it would be 
arbitrarily rounded up to 0. Although this method has been used by others,° 
we consider it unsatisfactory and only use it occasionally as the first step in an 
" iterative calculation of likelihood estimates. 

We shall discuss other theoretical aspects of the parameter estimation prob- 
‘Jem as the need arises in treating particular cases. This section is intended only 
to lay the theoretical foundation; the following section will give some numerical 


examples of estimation. 


TIilustrative Estimation Examples 


Our first example is intended to illustrate the likelihood estimation method 
for a simple case. Assume that the experimental data are: 


Ted, fo 1, 13. 2, 
ae Sel, Sous 


Assume further, that p:(1) = p2(2) = ps(3) = 14. Inthiscase, from Equation 
19, we have the following likelihood function: 


Pin = (: a my) Lt Se + (1 — 3w)(1 — |. 
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To find the largest value of p12: we consider: 


| {Op mae ay mG. ext 
Uist eae 5 (w — 1)? + c(2 — 3w), 


0 =eeet et iiee aor 


Consequently, relative extrema exist when either: 
(w = 1; ¢i=10) or (w = 144, ¢ = 1). 
The latter values yield the larger likelihood, namely: 
pin(w = 14, ¢ = 1) = 9. 
The restrictions of 20 further require: 
b+ce21, b>0, so b=0. 


Finally, since a = 3w — 2b, we have as our tentative maximum-likelihood esti- 
mates: 


Ce 6 = 0, ¢ = 1. 


However, as we shall now see, these values are not the final maximum-likelihood — 
estimates; the correct values are determined by checking the likelihood func- 
tion values on all the boundaries. 

The boundaries of our parameter region, as required by 20, are: 


a-= 0, b= 0, c= 0; 
i. b= 0, c= 0; 
a= 0, b+c=1; 
1 b+c=1. 


a 
I 


a= 


We consider these four cases in turn. In the first two cases, since w = (14) 
(a + 26), we have 

pin = 34, 26. 
In the third case, using 1 — c = b = (3w/2), we have: 


pian = (14) (6w* — Ow? + 2w + 1). 


This takes on the value 14 for w = 0 and w = 24, the boundaries on w when | 
a = O, and the interior maximum occurs when: 


On, Boal = (1/2)(9w? — 9w +1) or we Ne 


It is easily verified that pin < 34 when w = (3 — 1/5)/6. A similar analysis | 
of the fourth case also yields a value of pin less than 34. Thus, the maximum- 
likelihood estimates are @ = 6 = ¢ = 0. Exactly similar calculations, for the 


| 


| 
« 
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outcome patterns (s; = s2) and three responses in a single trial, yield the follow- 
ing maximum-likelihood estimates in the five cases. 


fin = 1 loko (el. 6 = 1, C—O: 
pur = 4 for @=0, §=1, é=0. 
fin = 3% for ¢@ = 0, b= 0, C= (). 
Pet 4/3/60 for f= 0, bleh /3/ C0: 
prs = 34 tor a= 0; b= : é = 4, 


It is apparent, from these sample calculations, that explicit solutions for 
maximum-likelihood estimates could be obtained in a straightforward but very 
tedious manner for trials with more than three responses. 

We shall now illustrate the use of our estimation methods by treating two 
synthetic examples, one of which consists of many repeated trials of three re- 
sponses each, and the other of which consists of one trial with 75 responses. 
The examples are ‘‘synthetic” because the data for them were obtained by 
choosing arbitrary values for the behavioral and state parameters, fixing also 


- upon an arbitrary choice of outcome sequences, and then calculating responses 


that would result by using a table of random numbers. Since this technique 
of using synthetic data is employed frequently in our work, we shall digress for 


: a moment to show how this is done. 


Consider an example of a synthetic trial having four successive responses, 


_ the first two of which have the same outcome and the third a different outcome. 


Our preliminary data are: 


m= 3 n=2; s=se=1, 53 = 2; 


ai(1) = po(1) = ps(1) = 4; q@ = 0.11, b' = 0.01, (22) 
@= 040, ¢ = 0.19, = 0.01, 2 = 0.50; M= 4. 
We next choose four from a table of random probabilities, say: 


R, = 0.862, Re = 0.283, R; = 0.622, Ry = 0.461. 


_ The first response is, therefore, 7, = 3 since p:(1) + po(1) < Ri. The prob- 
: ability vector, after the first response, is now: 


0.40 0.59 0.445 0.33 0.478 
p(2) = M*p(1) = |0.59 0.40 0.445].0.33 | = | 0.478 
0.01 0.01 0.11 | 0.34 0.044 


j Consequently, r2 = 1 since Re < pi(2). The new probability vector, after 
- the second response and outcome is: 


0.11 0.01 0,01 | 0.478 0,058 
p(3) = Mp(2) = |0.445 0.40 0.59 | 0.478] = | 0.430. 
0.445 0.59 0.40 | 0.044 0.512. 
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Again, since p:(3) + p2(3) < Rs then rs = 3. Continuing: 
0.50 0.49 0.405 || 0.058 0.447 
p(4) = M*(3) = [0.49 0.50 0.405] 0.430] = | 0.451] , 
0.01 0.01 0.19 ||0.512 0.102 


and, since pi(4) < Ri < pi(4) + p2(4), we have finally 7» = 2. This illustrates 
the method of generating synthetic trials. 

An IBM 704 Computer was used to generate 600 synthetic trials of three 
responses each with the following preliminary data: 


m=3; n=2; s=se=1; pill) = poll) = ps1) = %; 
a = 0.05, b' = 0.04, c = 0.50, 2 = 0.13, 
B= 0.01, c = 0.60; N = 3, gq = 600. 
When the 600 sequences were enumerated the results were as follows: 


Fin = 1, Fy. = 24, Fin = 262, Fix. = 16, Fy23 = 297, 


or, equivalently: 
fin = .00166, = fix = .04, —froxr = -43667, = fixe = 02667, — fins = .495. 
When these values for fj, are substituted in 21 we obtain: 

é = —0.0216, 6 =.0.0733, @ = 0.5329. 


This illustrates the fact that the estimates yielded by 21 cannot always be used 
without modification because, as in this case, they may violate the restrictions 
of 3. Of course, in this example, the true maximum-likelihood estimates would 
be the values of a, 6, ¢ that maximize the quantity. 


ln L(a, b, c) = In Pur +- 24 In Prue — 262 In Pin + 16 ln Pir -F 297 ln Pw, (23) 


where the p;;, are as given by 19 and a, b, c are subject to the restrictions of 3. 


These estimates are d = 0.0135, 6 = 0.0483, ¢ = 0.5454, forwhich In L(4, 6, 2) = 
—0.94655. : 


The actual 75 responses (7,) for a synthetic trial whose parameters were those 
of 22, produced by an IBM 704 run, are as follows: 


31321 32213 21213 23121 31233 21231 23231 31232 
32131 31332 32321-°31232° $2332°- 12121 23231 


The sequence of outcomes (s,) that was used in this synthetic trial was as 
follows: 4 


11212 42121 . 12204~ 19419) 2279919211 921475 11 ite 
VU2UE 24201" 12221 2A A 19218 an it 


In principle, but not in practice, one can easily find the maximum-likelihood 
estimates that follow from these values of p(1), 71, and s, by maximizing L 


Ree afm 
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in 8. We shall now describe and illustrate a method that yields approximate 
values for such estimates. 
Approximations good to one decimal place, for each of the six parameters, 
could be obtained by computing the value of the likelihood function 8 for each 
of the 11° possible sets of parameter values between 0 and 1; actually, there 
are somewhat less than 11° possible sets because of the restrictions that 
b+ cl < iand’?+ ¢ <1. Then the set of parameter values yielding the 
_ largest likelihood value would be chosen as the approximate maximum likeli- 
_ hood estimated parameters. The IBM 704 code we have used for such calcu- 

lations computes these values at the rate of about 11/sec., for the case repre- 
_ sented by the data of 24 and 25, including the successive choice of sets of 
_ parameter values and selection and printout of the largest likelihood value 
found for all sets. At this rate of calculation, it would require some thirty 

hours of IBM 704 time to determine the six parameter values accurately to 
_ about one decimal place, for a single trial consisting of 75 successive responses 
_ among three alternatives; obviously, some more economical scheme of calcula- 
_ tion must be found than simply computing and comparing among all points of a 
fine lattice in the parameter space. 

Two schemes have been used for handling data like that of 23 and 24. One 

_ consists in computing a relatively coarse lattice, perhaps starting with 3° points, 
and then computing a somewhat finer lattice centered around the best point 
_ found previously, until all points of the lattice give about the same likelihood 
~ value. Another method consists in searching directly for the best parameter 
_ values by some computational scheme that tries a point, or points, and then 
- moves next to a new trial point according to some systematic criterion. We 
shall call the first method the “successive lattice method,” and the second 
- method the ‘‘direct search method.’ We shall now present some results ob- 
tained using a successive lattice method. 
~ An IBM 704 code was written to make possible a lattice computation over a 
_ subspace of the 6-dimensional unit cube that represents the parameter space. 
This “lattice subspace,” and the fineness of the “mesh” within it, is chosen 
' by the experimenter; roughly speaking, the lattice subspace can include any 
ca closed interval for each of the six parameters, and the fineness of the mesh 
can be chosen arbitrarily and independently for each parameter. The follow- 
ing example, for data generated synthetically as for 24 and 26, will illustrate 
_ this procedure. 
_ For our present synthetic trial we chose: 


em=3, n=2; s,asin26; p(l)=p(l)=p(1)—; a= .01, 
2 bt = 0, ob =A, @ = 19, b= 0, @ =..25;.- N = 15. 


gy 
i The sequence of responses (r;) produced synthetically by the IBM 704 run 
were: 

3 At aed 32302912, 6 12323) 13131 21927 213238,-13232 
Boe ars) 31525 - 22013. "21213 2332123232 


26) 
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to take on the values 0, 14, and 1 until all allowable combinations were com- 
puted. The result was: 


In L(0, 0, 0; 14, 0, 0) = 75 (—0.7285) = In Li’, BY, c, @&, B, eo); 


’ 


Following this, the lattice in FIGURE 2 was used. The best result was: 
In L(0, 0, 0; 14, 0, 0) = 75 (—0.7207). 


Ficures 3, 4, and 5, with three additional lattices and their best results, were 
then used. 

The example was not carried beyond this stage, but it illustrates not only the 
difficulty in using the lattice method but also its usefulness. Thus, after com- 
puting for only about 3600 points (ten minutes of IBM 704 time), we have some 
idea of the location of the likelihood estimates and also we see that the value 
of the likelihood function varies little as the parameter values are changed 
rather substantially. We shall now turn to the direct search type of method. 


al ria a a? b2 c 
Min. 0 0 0 0 0 0 
Max. 1 1 1 1 1 1 
Mesh 2 2 2 2 2 2 
FIGuRE 1 
a bt ol a? 52 c2 
Min. 0 0 0 0.25 0 0 
Max. 0.5 0:5 OS 0.75 0.5 0.5 
Mesh 2 2 2 2 2 2 
FIGURE 2. 


There are a number of direct search methods that could be used. The steep- — 
est ascent methods are a popular variety for this type of problem. One of — 
these methods simply starts at some more or less arbitrary point in the parame-/ | 
ter space, then computes the value of the function on some fairly small lattice ) 
centered on this point, then determines from these points the direction in the 
parameter space that ascends most rapidly, then moves step by step in this 
direction until progress upward ceases, and then repeats this general process 
until a relative maximum seems to have been reached. Unfortunately, the 
method of steepest ascent does not seem to work well in our type of problem, _ 
where the function is a polynomial of very high degree in six parameters, so 4 
that there is no assurance that a relative maximum is also a general maximum. | 
We favor, instead, a method like that of Hooke and Jeeves," in which the 
ascent is more direct and is done without seeking the most rapid ascent at each 
move. ‘Their method was coded for the IBM 704, and we have modified this 
code for our present purposes (Appendix B). Its use will be illustrated and 
discussed in later sections, both for seeking maxima in cases like those just 
treated by the lattice method and in cases like those represented by 23. 


. 
‘ 
| 
a) 
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; We turn next to a few questions that arise in considering the statistical sig- 
nificance of some of the results obtained by using maximum-likelihood esti- 
mates derived in the manner just described. 


Parameter Significance 


In the preceding section, two sets of synthetic trials were analyzed to esti- 
_ mate the parameter values that produced them. In both instances, the esti- 
~ mates were apparently not very close to the true values. Our present task is 
_ to develop a significance test that will indicate whether or not such discrepan- 


a bl ol a? b2 c2 
7 Min. 0 0 0 0.125 0 0 
Max. 0.25 0.25 0.25 0.375 0.25 0.25 
A Mesh 2 2 2 oD 2 2 


In L (0, 0, 0; 0.375, 0, 0) = 75 (—0.7191) 


. Ficure 3. 

a a bh a a b2 c2 

ca Min. 0 0 0 0.31 0 0 

% Max. 0.12 0.12 0.12 0.43 0.12 0.12 

se Mesh 2 2 2 2 2 2 

= 

4 In L (O, 0, 0; 0.31, 0, 0) = 75 (—0.7184) 

“ 

Tv 

v Ficure 4. 

Be ai bl ol a? b2 x 

Min. 0.001 0.001 0.001 0.28 0.001 0.001 

Max. 0.061 0.061 0.061 0.34 0.061 0.061 
Mesh Z 2 2 y 2 2 


In L (0.001, 0.001, 0.31; 0.31, 0.001, 0.001) = 75. (—0.7190) 


FicureE 5. 


‘cies may reasonably be attributed to random fluctuations in our experiments. 
Such tests will also show how confidence intervals may be chosen for parameter 
estimates obtained in this manner. 

In one case, that of 600 trials each of three responses, the true values and the 
estimates were as shown in TABLE 1. The first significance test is directed 
‘toward a comparison of the observed and theoretical distributions, as shown 
in TABLE 2. 

- From the data of TABLE 2 we find x? = 5.12, indicating a degree of dis- 
‘crepancy between theoretical and observed distributions that would arise less 
than 10 per cent of the time; actually about as large a discrepancy as we would 
‘want to tolerate routinely. From a confidence region standpoint, we would 


ia 
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therefore say that a set of parameter values that led to a distribution with no 
larger discrepancy than this from the observed would be included within a 
confidence region based on the probability level corresponding to x? = 5.12. 
In particular, we would want our parameter estimates (0, 0.07, 0.53) to satisfy 
this criterion, and we now verify that this is a point within our confidence 
region; the new “theoretical” values in TABLE 3 are based on this set of parame- 
ter values, rather than upon the true values used in the synthetic run. From 
the data of TABLE 3 we find x? = 1.7, indicating that our estimates are well 
within the critical region determined by our data. Here then is one general 
method for determining points within the critical region. 


TABLE 1 
TRUE VALUES AND ESTIMATES 


a 6} | o a® 52 c 
True 0.05 0.04 0.50 0.13 0.01 0.60 
Est.* 0 0.07 0°53 None 


* These are not really maximum-likelihood estimates. 


TABLE 2 
OBSERVED AND THEORETICAL DISTRIBUTIONS 


piu pus pin pir pies 
Theoretical* 0.00175 0.04160 0.44900 0.04300 0.46500 
Observed 0.00167 0.04000 0.43667 0.02667 0.49500 


* Based on the true parameter values. 


TABLE 3 
THEORETICAL AND OBSERVED PARAMETER VALUES 


pur puz pir pi pi23 
Theoretical 0.003 0.044 0.433 0.035 0.487 
Observed 0.00167 0.04000 0.43667 0.02667 0.49500 | 


J 


It is already clear that a run of 600 trials does not narrow the critical region 
down to two-decimal accuracy for our parameters in this particular case. We 
turn now to our second type of estimate, that based upon a run of one trial with 
75 responses among the three alternatives, as displayed in 26. 

We have already noted that: 


In L(O, 0, 0; 0.31, 0, 0) = 75 (—0.7184) = In L(est. parameters), 


is the largest log-likelihood value obtained from the sequence of lattices used 
in approximating the maximum-likelihood estimates in our 75-response trial. 
A calculation made with the IBM 704 shows also that: 


In L(0.01, 0, 0.4; 0.19, 0, 0.25) = 75 (—0.7595) = In L(true parameters). 


ade 
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We consider the likelihood-ratio: 


Lest.) 
L(true) 


This value of x? is well within our confidence region, taking into account the 
six degrees of freedom represented by the six parameters. 

In general, then, we will consider a set of parameter values @ to be within our 
confidence region, relative to a maximum-likelihood estimate denoted 6, pro- 
vided the following condition is satisfied: 


2nd = 2Iln L@) — n L6)| = x2 < x,2, 


2nd = 2in| | = 6165 = xt 


where Xo is chosen so that the probability of observing no larger value is 0.95 if 
§ were correct, if we work at the 5 per cent confidence level. For example, if 
- an experimental trial with one animal yields maximum-likelihood estimates 6, 
and the likelihood value for another animal’s data, computed using the parame- 
ter value 6, is L*, then we will not consider the two animals to differ (on the 
basis of this experimental data) unless 


2[In L* — In L(6)| > x? 


In other words, if we assume that 6 estimates the true value common to the 


_ two animals, then this test will reject the hypothesis only if the second animal’s 


4 experimental data yield a likelihood value incompatible with this hypothesis. 
_  Anexample will illustrate this principle. Two rats were run in a three-choice 
- experiment with the following results: 


Rotana 1a2o2 15132 32132 12132, 12321 32121) 31321 31312 
12213 21211 21213-13132 13131 23232 1313 

Sa BD oe i A i 14041 11idierilitt A111! 41114 
NE 2 Fe Sal SO OB a GE WG Ee 

pee 95015 13225 (21521 32321 32313 21311 21311,-32131 
S22 desis 213.139 921313 3131525132)" 13132 

Reet i iii ieet2oit 1i1i2 22011 da2tt 2211) 11111 
Piette 22a tilt Jil iieet22t ie 21ii 


a After several IBM 704 mesh runs, the following approximate values were found 
for the six parameters for Rats 1 and 2: 


Rat 1: (0.06, 0, 0.4; 0.13, 0, 0.6) = 6. 
Rat 2: (0.09, 0, 0.5; 0.13, 0, 0.4) = 62. 


If we take x,2 = 12.6, corresponding to a 5 per cent confidence level based on 

six degrees of freedom, then the confidence region for Rat 2 will include any set 
of parameters @ such that 2[/ L(62) — Im L(6)| < 12.6; in particular, this con- 
dition will test whether Rat 1 is distinguished from Rat 2 when we use 6 = 41. 
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In this case, we found [2 lm L(62)] = —140.46, so our condition becomes 
[—In L(6) < 76.53]. Actually, —/n (6,) = 73.09, so that Rat 1 is indistinguish- 
able from Rat 2 on this basis. 

We also found several sets of parameter values within the confidence region, ~ 
so defined, for Rat 2. Among these are: 


b' = 0, a? = 0.13, b? = 0, ce = 0.4, 
and 
a': 0.08, 0.09, 0.08, 0.07, 0.09, 0.08, 0.07, 0.06, 0.05, 0.04; 
370.5, 0:4,°04, 0.4,°0.3,; 03,705, 95,505,208) 
Two sets outside the confidence region are: 
a! = 0.03, 0.02 


and b' = 0, 2 = 0.13, # = 0, 2 = 0.4. 
Gh ==" 0.30.5 


On another trial, for Rat 2 with 51 responses, the parameter estimates were as 
follows: 


6 = (0.06, 0, 0.3; 0.13, 0, 0.6) with —2 In (6) = 83.844. 


These parameter values seem very close to the ones listed above for Rat 1 so, 
on the basis of these experimental data, the two rats would not seem to be 
distinguishable. When these parameter estimates (6) for Rat 2 are used to 
determine the likelihood based on the previous 75-response trial for Rat 2, we | 
find —In (6) = 74.37 < 76.53, so that again we would not distinguish any 
difference in Rat 2 between these two trials. 

The examples just discussed should suffice to illustrate our methods for es- 
timating parameters, and for assessing the significance of differences among 
them. Basically, it is familiar maximum-likelihood estimation with confi- 
dence regions based on the likelihood-ratio significance test. 


Preliminary Experimental Results 


Some preliminary experimental results have been obtained with two rats,’ 
here called Rat J and Rat J. Unfortunately, Rat J died before the planned _ 
schedule of experiments had been completed. The full schedule has also not 
yet been completed for Rat J. 

Two types of experiments were conducted. In one type (Schedule A), a 
probability of reinforcement was assigned to each alternative, and the number 
of reinforced responses depended upon the actual responses made by the ani- | 
mal. In the other type (Schedule B), the sequence of reinforcements was. 
entirely independent of the actual responses made. 

Schedule A used the probabilities (0.2, 0.6, 0.8) or (0, 0.6, 1). Schedule B 
normally consisted of repeated blocks of 12 stimuli, each block starting with six 
reinforced responses and ending with six unreinforced responses. Normally, an 


animal was kept responding for about one hour and made 100 to 350 responses 
in that period. 
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The animal could press any one of six bars at each response. If the animal 
moved to the center of the cage so as to activate the photocell, after pressing 
any bar, then all six bars were reset and made active. The apparatus recorded 
the time at which each bar was pressed and the time when the photocell was 
first activated after each bar-press. For the present analysis, the responses 
used were the bars first pressed after each resetting by the photocell; correspond- 
ing to each such response 7; , there is either an outcome s; = 1, denoting that 
the response was reinforced, or an outcome s, = 2, denoting that the response 
was not reinforced. Also, for the present analysis, bars symmetrically op- 
posite within the cage were treated as though they were identical; thus, there 
were three alternatives available for each response: a left bar, a middle bar, 
and a right bar. 

Every effort was made to keep the opposite sides of the cage as nearly iden- 

_ tical as possible. For example, the walls behind the bars were painted differ- 
- ently for left, middle, and right, but the same for each direction on both sides 
_ of the cage. Nevertheless, it became apparent that the experimental animals 

TABLE 4 
Maximum-LikeLinoop EstTIMATEs 
2 Starting State Assumed 


s 


a Trial No.| Rat a my a @ & tes N Ds d 

: 

4 144 J 0.999 | 0.085 | 0.915 | 0.949 | 0.007 | 0.987 | 231 | 0.772 | 0.036 

ta 146 df 0.949 | 0.101 | 0.899 | 0.949 | 0.151 | 0.849 | 134 | 1.017 | 0.063 

‘4 148 J 0.999 | 0.063 | 0.931 | 0.999 | 0.033 | 0.967 | 269 | 0.829 | 0.031 

150 | JF | 0.899 | 0.100 | 0.900 | 0.999 | 0.100 | 0.700 | 144 | 1.064 | 0.058 

+ 196} if 0.999 | 0.001 | 0.998 | 0.999 | 0.050 | 0.949 | 206 | 0.944 | 0.041 

y | 

4 { Search calculation not complete. 

e 

: behaved differently on opposite sides of the cage, so grouping together two bars 
: ‘symmetrically placed in the cage proved to be a more arbitrary combination of 
_ response classes than had been intended. Consequently, the present analysis 
4 is offered as an example of results obtained with a somewhat arbitrary grouping 


of response classes, rather than for three response classes that might more 
reasonably be expected to satisfy the symmetry conditions upon which our 
- mathematical model is based. 
_ The results shown in TABLE 4 were obtained using Schedule A (0.2, 0.6, 0.8) 
- with Rat J , and using Schedule B with Rat J. The quantity d, in TABLE 4, 
is the relevant value for x,2 working at the 1 per cent level with six parameters; 
that is, any set of six parameters yielding a likelihood value less than (L* + d) 
is within the 1 per cent confidence region. (In the table, LA = (in L\/N ) 
For example, on Trial 148, the following six sets of parameter values include 
three sets that are within this confidence region (L* < 0.86) and three sets that 
are not. , 
The first three sets of parameter values shown in TABLE 5 differ from the 
-maximum-likelihood estimates, shown in TABLE 4, by the following maximum 
‘amounts: 0.050, 0.100, 0.131, 0.100, 0.072, and 0.072. It is likely that parame- 
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ter values could be found within the confidence region that would show even 
greater deviations. On the other hand, the second three sets of parameter 
values shown in TABLE 5 differ from the maximum-likelihood estimates, shown 
in TABLE 4, by the following minimum amounts: 0, 0.013, 0.031, 0, 0.018, and | 
0.032. 

Thus only rather slight changes in certain parameter values will sometimes 
place the set outside the confidence region. It can only be concluded, from 
the data of TABLE 4, that the maximum-likelihood estimates are reasonably 
similar from trial to trial and from rat to rat; a more extensive experiment and 
analysis is required in order to test this apparent stability of parameter esti- 
mates and to narrow the confidence regions appreciably. 


TABLE 5 
TriaAL 148 LIkELIHOODS 


ai bt a a? b2 ca 1 Bog! 
0.949 0.100 0.900 0.999 0.052 0.948 0.860 
1.000 0.200 0.800 0.899 0.105 0.895 0.858 
0.999 0.100 0.900 0.999 0.001 0.898 0.848 
0.999 0.100 0.900 0.999 0.001 0.999 0.920 
0.999 0.100 0.700 0.898 0.102 0.898 0.912 
0.999 0.050 0.900 0.999 0.051 0.898 0.878 
TABLE 6 


Maximum-LikELIHoop ESTIMATES 
Starting State Predicted 


a 


Trial | pea) | ped) | oan | bi a a be ée L* 


146 | 0.215 | 0.739 | 0.046 | 0.937 | 0.112 | 0.875 | 0.987 | 0.086 | 0.890 | 1.010 
148 | 0.331 | 0.562 | 0.107 | 0.993 | 0.070 | 0.924 | 0.999 | 0.026 | 0.974 | 0.822 
150 | 0.080 | 0.356 | 0.564 | 0.942 | 0.076 | 0.918 | 0.999 | 0.063 | 0.880 | 1.056 — 


The maximum-likelihood estimates, shown in TABLE 4, were all computed! 


| 


on the assumption that the first response of each trial was equally likely to be _ 


any one of the three bars, thus: ~pi(1) = po(1) = p3(1) = 14. The computa- 
tions were done on the IBM 704, using the modified search code described in 
Appendix B. 

Another calculation was made, also using the search code, in. which the initial 
state for each trial was taken to be the ending state for the preceding trial; _ 
this is an alternative to the assumption of equal probability among bars at the — 
start of each trial. The results are shown in TABLE 6, It is clear, from TABLE 
6, that the use of predicted starting states improves the fit of the model to the 
data, since each value of L* is smaller than in TABLE 4, where the starting 
state was assumed to be probability 14 for each bar. Unfortunately, the esti- 
mates of TABLES 4 and 6 are not entirely reliable because the search code does 
not absolutely ensure correct likelihood values. 


ec as 


a 
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The starting states shown in TABLE 6, for each trial, were calculated from the 

data for the preceding trial. More specifically, the starting state for Trial 144 
was taken arbitrarily to be pi(1) = p.(1) = p3(1) = 14. This yielded the 
maximum_-likelihood estimates for Trial 144, as shown in TABLE 4. The end 
state for Trial 144, based upon the estimated parameters, was calculated to be: 

_ px(232) = 0.215, pe(232) = 0.739, pe(232) = 0.046. Since the first response 
_ in Trial 146 is the next one made by Rat J after the 231st response in Trial 144, 
_ with a rest of one hour intervening, the end state of Trial 144 was taken as the 
starting state for Trial 146. Similar reasoning applies to Trials 148 and 150. 
The actual schedule of reinforcements used on Trials 144, 146, 148, and 150 
led to the gross results shown in TABLE 7. The results in TABLE 7 followed from 


== TABLE 7 
k DISTRIBUTION OF BAR PRESSES 


Number of times bar pressed 


- Trial No. Reinforced Unreinforced 

eo Total 

E- Left Center | Right Left | Center | Right 
144 231 36 128 Z, 24 31 10 
146 134 12 36 9 40 28 2) 
148 269 3 97 60 29 67 13 
150 144 38 18 20 10 48 10 

TABLE 8 


) A r 
=a 
a, 
ie 
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COMPARISON OF ACTUAL AND SCHEDULED REINFORCEMENT PROBABILITIES 


Scheduled Actual 
Trial No. 
L c R L Cc R 
144 0.6 0.8 0.2 0.60 0.81 Onli 
146 0.2 0.6 0.8 0523 0.56 0.50 
148 0.2 0.6 0.8 0.09 0.59 0.82 
150 0.8 0.2 0.6 0.79 OE 0.67 


use of Schedule A (0.2, 0.6, 0.8), with a permutation of the probabilities among 
the bars after Trials 144. and 148. The actual relative frequencies are compared 
with the scheduled probabilities in TABLE 8. The starting states shown in 
_ TABLE 6 clearly show the tendency for the rat to select the responses that have 
led to reinforcements most regularly during the recent past, but they also show 
that former good alternatives continue to be selected for a considerable period 
- of time. ‘ ; ws 

It is interesting that (61 + @) and (@ + ¢) are close to unity in value for 
each trial shown in TABLE 6. Furthermore, if the combining-of-classes assump- 
tion holds, then necessarily 


cai Sf b= ¢) = 0fors = 1,2 
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The values for C* actually observed are given in TABLE 9. Although the ob- 
served values for C* are all near zero, our experiments and analysis are not 
extensive enough to determine whether or not the combining-of-classes assump- 
tion is acceptable for this test situation. This question is being investigated — 
further. 

It now seems that one trial, with V = 1000, is about right to yield parameter 
estimates accurate to two decimal places. With our present computational 
methods, it requires about 30 min. of IBM 704 time to obtain two separate 
sets of parameter estimates for one such trial, using the search code described 
in Appendix B. For such a trial, the confidence region, at the 1 per cent level, 
has a value of d < 0.01; this seems to be precise enough to yield reasonably 
sensitive comparisons between sets of parameter estimates. 

The effect of small changes in parameter values, in the general neighborhood 
of values observed for Rats J and J, is being investigated by the use of syn- 
thetic runs on the IBM 704. This synthetic technique is being used to explore 
empirically the distribution of ZL, and to explore changes induced by changes 
in the experimental situation. In other words, the synthetic rat also becomes a 
subject of experimental investigation and various comparisons between gross 


TABLE 9 
OBSERVED VALUES FOR C* 


Trial No. 144 146 148 150 196 
Cc} 0 0.018 —0.003 0.023 0 
C —0.019 —0.018 0 —0.007 0 


behavior of synthetic rat and actual rat should help to highlight similarities and 
differences between a real rat and its mathematical model. 


Conclusions f 
(1) Stimulation through electrodes implanted in the hypothalamic region of , 
a rat’s brain can be used efficiently to provide reinforcing stimuli in a learning 
experiment extending over a period of months. : 

(2) Maximum-likelihood estimates of the six parameters in the symmetry 
model can be approximated, using a search code, in less than 30 min. of IBM 
704 time, for a trial consisting of 1000 successive responses from among three 
alternatives. 

(3) Estimated values of the six symmetry model parameters are reasonably 
constant from trial to trial for a single rat, when measured in an experiment in _ 
which the rat has three alternatives for each response and the schedule of re- 
inforcement depends upon the responses actually made. | 

(4) The very limited experimental data analyzed are compatible with the 
hypothesis that estimated parameter values are the same for the two rats 
tested, even though the parameters are measured using two quite different 
types of reinforcement schedules. 


; (S) The data analyzed are also compatible with the hypothesis that the com- 
bining-of-classes condition is satisfied. 
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APPENDIX A 


Experimental Procedure 


Subjects and electrode implantation. The majority of the experimental work 
was carried out with two male, mature albino rats. The electrodes and im- 
plantation techniques used were those describe in detail by Olds and Milner.” 
In brief, the electrodes are a pair of twisted silver wires, diamel coated for 
insulation except at their tips, thus restricting the stimulus to that point. Each 
pair of electrodes is secured to a lucite pedestal that, in turn, is held in place 
on the skull by screws through the skull bone. This technique has proven to 
be very satisfactory for longitudinal studies using implanted rats. 

The area selected for implantation was the medial forebrain bundle, lateral 
to the anterior hypothalamus, an area Olds has rated as positively reinforcing.” 

Experimental apparatus and procedure. ‘The reinforcing stimulus was an AC 
pulse of 10 to 25 wamp. intensity, with duration a function of the length of time 
the bar was depressed, up to an automatic cut-off time of 0.6 sec. Stimulus 
threshold values for reinforcement in this area are about 12 wamp., while cur- 
rents in excess of 25 wamp. are best avoided since they make the rats hyper- 
excitable, disrupting their performance in a choice situation. 

Following a one-week postoperative recovery period, the rats first were 
trained to bar press for a stimulus in an enclosed box with a single bar. The 
response was shaped up in the usual manner of giving reinforcements for suc- 
cessively closer approximations of the desired response, until the bar finally is 
depressed. Thereafter reinforcements are given automatically after each bar 
press, a response both rats were making at rates of 500 to 1000/hour within a 
few hours of training. 

The testing of rats in a choice situation necessitates the rat’s return to a 
neutral choice point between responses. There are advantages to having them 
proceed to a choice point on their own, so the rats next were trained to go to 
the opposite side of the box after every bar press. This more complex response 
also was shaped up by the method of reinforcing successive approximations to 
the required response sequence. When they had learned to make this total 
response with a minimum of errors (usually within 10 hours of experimental 
sessions), they were advanced to the final training phase. 

The first apparatus used in the final phase was a large circular cage with three 
bars equally spaced around its circumference and a disc in the center marking 


-_ 


the choice point. Although the rats readily transferred their previous learning 


to this three-bar condition—going to the choice point after pressing any of the 
three bars—they also demonstrated a strong bias that practically reduced this 


to a two-choice situation. Upon reaching the center choice point, they showed 
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a strong preference for choosing their next response from between the two bars 
facing them from across the cage, rather than making the inefficient response of 
turning around to return to the bar just pressed. Reinforcing the bars differ- 
entially may have overcome this bias, but it seemed preferable to start with a 
less biased three-choice situation. 

Therefore, a new apparatus was designed to incorporate two desirable fea- 
tures: (1) a choice point at which the rat’s visual field would include all of the 
bars from which he is expected to choose; and (2) ease in converting the ap- 
paratus to test the rats on any number of bars. The apparatus is illustrated in 
FIGURE 6, and consists basically of two fan-shaped center sections, the sides of 
which can be pivoted at the center point to encompass the desired number of 
removable U-shaped panels at each end. When the rat passes through the 
photocell beam at the choice point, the stimulus circuit is reset so he can be 


Photo- 
cell 


Choice 
Point 


FIGURE 6. 


reinforced for his next press of any of the bars. Control circuitry makes it 
possible to run experiments using patterned or relatively random partial rein- 
forcement schedules, with either the same or differential schedules on each bar. 
In the initial sessions in this apparatus, the rats were run with one bar at 
each end, but they adapted so quickly to this new condition that in subsequent 
sessions they were run with three bars. Under conditions of 100 per cent rein- 
forcement, there were very few failures to return to the choice point between 
bar presses. Under partial reinforcement schedules, failures to return in- 
creased, indicating emotional outbursts rather than lack of training. The 
reversal of differential reinforcement schedules has pointed up one problem 
with this apparatus: a possible nonequivalence of the bars at both ends. _De- 
spite our attempts to eliminate all differentiating cues, the rats are discriminat- 
ing sufficiently between the ends to select different bars at each end. If this 
nonequivalence proves to be the case, the apparatus will be modified by - 
taining the bars at one end only. At the other end the bars will be ine 
by a blank panel attached to the center section to form a cul-de-sac, with the 
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photocell relocated close to the end of this blind alley. This will force the rat 
to enter the alley to reset the device, so that on leaving the alley he can again 
face all of the bars from which he is expected to choose. 


APPENDIX B 


Search Codes 


Maximum-likelihood estimation and many other situations require an effi- 
cient computational procedure for determining the location of the largest (or 
smallest) value of a function of several real variables. Some or all of the vari- 
ables may also be restricted to some region; common restrictions would be non- 
negativity or limitation to the unit -cube. 

Our maximum-likelihood estimation problem is essentially that of finding 
the values for a set of real variables that maximizes the value of a polynomial 
in the variables, where the variables are restricted to a specified portion of the 
unit cube. 

Mathematically, if we let « denote the set of real variables (x1, x2, +--+ , Xn), 
f(x) a real function of the variables x, and R the domain over which the vari- 
ables are allowed to range, then our problem is to find a set of values « such 
that f(£) > f(«) for all x in R. Or, rather, our need is for a computational 
procedure that will solve this problem economically when f(x) is one of the 
likelihood functions encountered in analyzing experimental data. 

When f(x) is a polynomial, given explicitly as is ours in the likelihood estima- 
tion situation, there is no theoretical difficulty in finding the required values « 
by the ordinary methods of calculus. In brief outline, for the case when R 
is the unit m-cube, all that is necessary is: 

(1) Find the roots of the polynomial f’(x) = df(«)/dx. For each such root, 
if it is in R, test to determine whether or not it yields a relative maximum for 
f(x). Let M = f(x) be the largest of these relative maxima. 

(2) Let X = (Xi, X2, +--+, Xn), where X; is either x;, 0, or1. Then f(X) 
is the value of the function on some bounding hyperplane of the unit »-cube; 
the dimension of this hyperplane is the number of x; included in X. By the 
method outlined in (1), the largest value of f(X) can be determined. This 
includes the case where all X; are 0 or 1, representing the values at the corners. 
Let M(X) = f(X) be the largest of the relative maxima for the bounding hyper- 
plane defined by X. 

(3) The required value of £is the X that satisfies the condition 


f(X) > M(X) for all X. 


In other words, the locations of the largest interior relative maxima are deter- 
mined on all bounding hyperplanes, and the solution to our problem is the one 
that yields the largest value of the function. When the degree of the poly- 
nomial is high, or the number of variables is large, the techniques of the calculus 
are generally too inefficient to permit their use, as is normally the case for our 
likelihood functions. Consequently, we have turned to the approximation 
method to be described now. 


Hooke and Jeeves" have developed a procedure for searching through the 
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domain R, and have used the procedure successfully on various problems. 
Their procedure involves a combination of two rather different types of search- 
ing operations: they have called one type “direct search” and the other type 
“pattern search.” Only the central features of their procedure will be pre- 
sented here. 

Direct search starts with an arbitrary point «! in the m-dimensional domain K 
where R is defined by the inequalities: 


eee a De LOR at al? wren a: 


The first computational step is to compare f(x!) and f(ay! + A, wl, +++, a) 
where A is some positive real number and where x;' + A is changed to D, if 
vy'-+ A> D,. If the change in the variable yields a larger value of the func- 
tion then the new value is retained. If not, x;! is decreased by amount A, but 
not to a value less than d; , and the changed variable is retained if and only if 
the function value increases. The procedure is repeated with a2, then with «3 , 
and so on through x, . If no change in the variables yielded an improvement, 
then the entire process is repeated with (A/2) replacing A, unless (A/2) < 
Amin , until the process either terminates or an improved value of the function 
is located at some stage. 

Pattern search follows each instance of success in direct search. If x° de- 
notes the starting point of a direct search step, and « the improved point at the 
termination of direct search, then pattern search starts with a direct search 
from the point 2(4 — x°) = #°; as always, if any coordinate violates the con- 
dition 

Oy Seed Ke 82) SD; 


then it is replaced by the appropriate value of d; or D;. In this direct search, 
as part of pattern search, the value of /(«’) is ignored and comparisons are with 
f(#) at each step during pattern search; furthermore, the terminal value of 
(A/2*) in use at the termination of direct search is used unchanged throughout 
the direct search portion of pattern search. If direct search from #° yields a 
terminal point x’ such that f(x’) > /(*), then a new pattern search starts at 
the point 2(a? — %), and «° and % become the new values for * and «? in this 
pattern search step. On the other hand, if f(«?) < f(#) then pattern search is 
terminated and direct search is done about # with (A/29**) replacing (A/2?). 

Eventually the search terminates, when AQ) < Amin, and the desired 
value of # is approximated by the final value of x in the search process. — This 
brief description omits several important technical details, and says nothing of 
the actual computer codes used, for which the reader must be referred to the 
paper of Hooke and Jeeves.” 

The search code of Hooke and Jeeves was necessarily modified slightly for 
our maximum-likelihood problem, because the domain R is not of quite the 
same form as that treated by Hooke and Jeeves. There are two additional 
restrictions on the variables, in our case, namely: 
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Primarily for convenience in modifying the computer code, and because the 
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change seemed adequate to take proper account of these two added restric- 
tions, the “modified search code” actually used retained the altered value of a 
variable 6 (or c) and changed c (or 6) as necessary to satisfy the restriction 
b+c<41. For example, if b' is to be replaced by (6! + A) when (6!+ A < 1) 
and (b! + A+ c! > 1) then c! is replaced by (1 — 6!— A). The calculations 
reported upon in this paper, as using a search code, were all done with this 
simple modification of the Hooke-Jeeves code. 

It is evident that the search code does not necessarily and inevitably ter- 
minate at a point that is even near the correct one. Nor does it seem likely 
at present that any computational procedure will soon be found that is both 
reasonably economical and guaranteed to yield an approximate solution near 
the correct one. Consequently, all of our empirical results are open to question 
on this ground. 

We have sought protection against this danger of error in several ways. 
The three principal ways were: (1) comparison of search code approximations 
with lattice code approximations in sample instances; (2) comparison of search 
code approximations with others obtained using starting points as distant as 
possible from the previous solutions; and (3) checks for agreement of solutions 
by search code techniques with a few simple likelihood estimation problems for 
which exact solutions are known. 

Of course, the most important source of protection against such errors is the 
comparison of calculated parameters from among independent trials with 
different animals. The results reported in the present paper were subjected 
to all of these tests, except for the incomplete calculation of parameters for 
Rat J. 
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DECISION THEORY APPLICATIONS IN HUMAN 
DECISION MAKING 


Philip L. Stocklin 
Office of Naval Research, Washington, D. C. 


An important aspect of decision theory is its potential as a unifying analytic 
framework for several levels of human decision making. To introduce the dis- 
cussion, let us first look briefly at two applications, illustrative of quite differ- 
ent decision-making levels, that sprang largely from research-system needs of 
the United States Navy. 

About two years ago Cron and Martin,! at the Underwater Sound Laboratory, 
New London, Conn., tested a group of about 20 observers in two simple de- 
tection tasks, for the purpose of comparing their performance to that of a 
decision theory ideal observer (which we may call a “generalized’’ ideal ob- 
server). Such an observer is one who performs a likelihood ratio measurement, 
then compares the result of this measurement with a bias or threshold. Each 
of the two tasks required a binary decision; one task used a numerical presen- 
tation and the other, a graphic presentation. In both cases, the likelihood ratio 
measurement consists of taking an average. 

The numerical task was given in four parts. In Part 1, nine numbers were 
drawn at random from a population of normal deviates (the noise population) 
or from a similar population to which a small constant was added (the signal- 
plus-noise population). The nine numbers were listed on a card, and the ob- 
server was given the card and asked to decide which of the two populations the 
nine numbers came from. A run consisted of an equal number of cards from 
each of the two populations arranged in random order, and the observers were 
told this fact beforehand. Observers were also told that a correct decision was 
worth +1 and an error —1 (the so-called unit cost matrix). The difference 
between the two populations appeared visually marginal. As a more exact 
measure, the numerical signal-to-noise ratio was —6 db. 

Part 2 of the numerical task was, with two changes, identical to that of Part 
1. First, four numbers per sample were used instead of nine, and second, the 
S/N ratio was increased to —2.5 db. The net effect of these two changes is 
to leave the index of detectability, d’ (in the sense of Tanner and Swets*) un- 
changed from that of Part 1. 

The results of these tests are that in Part 2, when only four numbers had to 
be processed per sample, performance of the human observers was indistin- 
guishable from that of the generalized ideal observer. In Part 1, however, 
when nine numbers had to be processed, human observer performance was 
significantly poorer than that of the generalized ideal observer. This result 
suggests that, at least for the type of test attempted, d’ may not be an invariant 
of observer performance. 

Part 3 of the numerical task was identical with the four-number, equal a 
priori probability task of Part 2, except that cost ratios were altered by a factor 
of 4 from the unit costs used previously. As a result of this cost change, the 
ideal observer should use a much lower threshold to decide (S + N) than that 
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used in Part 2. Actual observers did, in fact, lower their threshold somewhat, 
but went only part way toward the ideal observer threshold. Interestingly 
enough, their d’ remained the same as that of the ideal observer, that is, their 
efficiency remained unimpaired. 

Part 4 was identical to the four-number task of Part 2 with a unit cost 
matrix; however, the a priori probability of occurrence of a sample from the 
noise-alone population and from the signal-plus-noise population was changed. 
Instead of being equally likely, this probability varied from run to run, up to 
a ratio of 4:1. As before, the observer was told what a priori probability to 
expect in any run, as well as costs and other facts of the experiment. The 
result of the Part 4 test is that the observers did not take a priori probability 
into account in selecting a threshold; furthermore, their average efficiency 
decreased: d’ went down. 

Finally, the same random numbers used in Part 3, above, were plotted as 
y-coordinates on a graph and connected with straight lines. Results of this 
visual averaging task were compared with the results of the corresponding 
numerical task, and no statistically-significant difference was found. 

These tests are of the same type as the more basic and comprehensive ones 
reported by Tanner and Swets (of. cit.), and the results quoted above do not 
appear to be contradictory to theirs. We note, however, the variability of 
observer performance with constant experimental d’, the relative sensitivity to 
cost variation and insensitivity to a priori probability variation, and the differ- 
ence in dependence of actual detectability upon cost versus a priori probabil- 
ity, despite their similar theoretical roles. 

This experiment is at the psychophysical level of application; let us examine 
briefly a second level of great practical interest, the real-life operational level. 

Here the problem we shall discuss is that of conflicting information or, in 
psychological terms, incongruent stimuli. I shall comment chiefly on the 
decision-theoretic model* that is more fully treated in an unpublished paper 
by Becker and myself.* 

The problem is to develop a model where a decision must be made, and in- 
formation comes to the final decision maker from several sources of differing 
validity. Such problems occur every day; the desired solution is a decision 
rule more-or-less independent of such particulars as the type of stimuli and the 
order of the decision. What is proposed is a model containing, as in a standard 
decision-theory formulation, one signal (or “real world occurrency”’) space, but 
that includes two (or more) observation spaces. Each point in a given ob- 
servation space is obtained from a point in signal space to which is combined 
a sample from a noise space. The observation spaces may be the decision 
spaces of inaccurate subdecision makers, who report what they think they see 
in the signal space to the final decision maker. The observation spaces may 
overlap and, in fact, may present data that, for all or most thresholds of the 
decision maker, would lead to different final decisions if considered separately. 
The likelihood ratios have been worked out for two such observation spaces, 
and have proven quite workable for comparing such situations as: (1) the final 


* Primarily the work of Gordon Becker of Yale Universi 
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decision maker knows noise contamination of one of the subdecision makers is 
low, but is not sure which observation space goes with which subdecision maker; 
or (2) the final decision maker knows which subdecision maker goes with which 
observation space, but both subdecision makers are highly contaminated with 
noise. 

Becker notes the flexibility in such a model and suggests, through the addition 
of feedback, a possible learning model. Immediate applications to group de- 
cision situations—such as conferences—are apparent. 

The purpose of juxtaposing a psychophysical experiment and this conflicting 
information model is severalfold. First, may I emphasize the capability of 
decision theory as a useful analytic tool at several behavioral levels. This 
usefulness can be traced to its statistically-complete character, in the sense of 
Woodward,‘ and to the distinct separation between three sorts of quantities: 
(1) the physical or psychophysical probabilities, that is, those probability dis- 
tributions that describe the physical stimuli; (2) the operational or a priori 
probabilities, that is, the probability that such-and-such a situation will occur; 
and (3) the costs or utilities, which usually represent numbers not derivable 
within the context of the situation itself, but rather weighting (stipulated or 
measured) given to various combinations of stimulus and response. 

One of the most significant contributions of the decision-theory approach to 
date is the decisive reduction it is effecting in the arbitrariness of psychophysi- 
cal criteria. In the purely physical and in the operational domains, especially 
when dealing with highly perturbed—that is, noisy—situations, reduction of 
criterion arbitrariness has really been essential to progress, and this may well 
be true in the psychophysical area. 

Both of the studies discussed here were strongly system-motivated. In the 
psychophysical experiments, for example, the earlier success of decision theory 
in measuring processing instrumentation, applied both in the literature and 
through the use of Monte Carlo digital-computer techniques, generated the 
need for similar analysis of the psychophysical component in navy research 
systems. From the system point of view, an identical language for all proc- 
essing components is absolutely essential, since system performance should be 
describable by the smallest possible set of numbers, preferably just one. 

We see, then, a strong unifying potential in decision theory, linking human 
and nonhuman signal processing. A second area of unification is the capability 
for accurate description of varied states of knowledge of the decision maker. 
The description is accurate in the sense that the effect of a state of knowledge 
upon the decision maker’s ability to make a correct (or least risky) decision 
may be accurately determined. In behavioral studies, such as the one men- 
tioned on conflicting information, an accurate state-of-knowledge description 
becomes an all-important capability. 

We may invert this last argument, however, and use the result to introduce 
the debit side of the ledger: that is, areas where decision theory is not completely 
satisfactory. ; ; a“ : 

Accurate description of a state-of-knowledge is equivalent to the statement 
of a probability distribution (physical and/ or psychophysical) over every space 
in a decision theory flow diagram; in addition, the so-called a priori probabil- 
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ities may have to be defined over a set of signal spaces. Statistically this 
means we have in decision theory a highly parametric tool the use of which de- 
mands either that we possess a rather complete statistical description of what 
is going on, or that we, in the absence of such knowledge, adopt some conven- 
tion equivalent to the assumption of such a state-of-knowledge. Although it is 
possible to conduct a sensitivity analysis for each and every situation concelv- 
able to us, such an approach only transfers the need for statistical description 
into the judgment of the analyzer, and so begs the question. 

One asks, then, about quasi-parametric or even nonparametric decision 
theory. Since I am beginning to come dangerously close to the province of 
mathematical statistics here, I should like to restrict my comments to just one 
promising avenue. One way to categorize decision theory studies is whether 
they are concerned with fixed sample sizes or whether they involve sequential 
analysis; that is, a sequence of sample-process-decision-sample-process-decision, 
and so forth, until a final decision can be made. The criterion for number of 
samples is that the (maximal) error probabilities associated, on the average, 
with the final decision will be less than certain specified numbers. Usually, 
processor design is taken as an invariant from step to step. But if this invari- 
ance is not required and, instead, one permits the processor to learn its design 
as it measures the input, in the sense of Hofstetter’s adaptive processors,® we 
have a technique that develops the statistics as it goes along. In fact, a useful 
transition from nonparametric to highly parametric theory may be possible in 
this manner. Clearly, many existing studies in learning theory and in percep- 
tion are quite pertinent; a generalization of sequential decision theory com- 
bined with the results of these studies appears to provide a method of attacking 
basic state-of-knowledge difficulties in decision theory applications. 

Closely connected with the state-of-knowledge difficulty is the practical re- 
striction to low-order and normal—or, at least, relatively simple—statistics. 
For very practical and sometimes decisive technological reasons, we are in- 
terested in delimiting possible system performance. If decision theory is used, 
such delimiting is almost always sensitive to the statistics used. With normal 
statistics and the accompanying paraphernalia, such as low-order Markoff 
process assumptions, much has and can be done. Once we move away from 
these assumptions, however, decision theory as an analytic tool becomes quite 
cumbersome, and we may well question its usefulness, not on the grounds that 
it is a poor model of reality, but merely on the grounds that we cannot solve 
the numerical computations that the theory implies. 

Intractable probability distribution functions are not the only source. of 
computational difficulty. It is noteworthy that almost all the applications 
successfully carried through to date involve very low-order decisions: on the 
order of two to four. That is, in signal detectability we are concerned with 
making a second-order decision—one of two possible choices—or in the sequen- 
tial detection problem, one of three possible choices. Since the number of 
likelihood ratios that must be computed grows roughly as the square of the 
order of decision, we may expect computational difficulties in high order decision 
theory, even for very straightforward statistics. 

Both complex distribution functions and high-order decisions point toward a 
useful analysis that is dependent primarily upon statistical parameters instead 
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of upon the distribution functions themselves, and upon gross measures of the 
decision set: that is, toward a direct weakening of the power of the theory. 
However, the difficulty of high-order decisions can suggest a most interesting 
line of further theoretical development. For example, under fairly general 
assumptions, the optimum processor for an uth order decision may be described 
as a collection of (m — 1)/2 likelihood ratio processors, followed by a large 
number of coincidence circuits. The coincidence circuits may be arranged in 
m levels, where m = 2". The output of this collection of likelihood ratio proc- 
essors and coincidence circuits is the set of permissible decisions, and in general 
the outputs of a number of coincidence circuits will go to the same decision. 
It is an interesting property of this arrangement that every likelihood ratio 
processor and every coincidence circuit is essential in the sense that, for every 
point in observation space, one and only one decision is to exist. Omission of 
just one coincidence circuit, for example, means that at least one point will 
exist in observation space for which no decision is possible. Several properties 
of this matrix appear quite similar to those of existing models in perception; 
however, the point I should like to bring out is that, while computational re- 
quirements of high-order decision theory may force us to use gross or average 
measures, this difficulty—as with the other difficulties—is often heuristic. It 
may, as in the case cited above, not only lead to unexpected and possibly use- 
ful gross measures, but it may also provide insight into techniques that have 
developed from wholly different points of view. 


Summary 


Having gone fairly far afield in this last discussion, I should like to stop and 
summarize the point of view developed in this paper regarding the application 
of decision theory. 

We have inspected, briefly, a psychophysical experiment and a behavioral 
model to bring out the flexibility and unifying potential of decision theory. 
We have seen that through clear separation of three types of measures—the 
psychophysical probabilities, the a priori or operational probability, and the 
cost—criterion arbitrariness may be reduced, but human observers do not al- 
ways follow one-to-one with the dictates of the theory. Continued successful 
application of decision theory in nonhuman components of research systems 
generates a need for a similar approach—or at least, a similar language—for 
human signal-processing performance. On the debit side, there are difficulties 
due to the parametric nature of decision theory, such as state-of-knowledge 
descriptions, and computational difficulties that arise in the use of other-than- 
normal statistics and in high-order decision theory. While these difficulties 
exist, in many cases promising directions of further research, such as adaptive 
processing from sequential decision-theory analysis, are indicated. In my 
opinion, we are at present only beginning to realize the usefulness of decision 
theory in systems applications involving human decision making. 
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SOME PROBLEMS IN VISION REGARDED WITH RESPECT TO 
LINEARITY AND FREQUENCY RESPONSE* 


L. H. Van der Tweel 
Laboratory of Medical Physics of the University of Amsterdam, the Netherlands 


Introduction 


In general there is a considerable difference between the methods used in 
psychophysical research on vision and those of electrophysiology. The work 
done on threshold vision can be considered the main contribution to our knowl- 
edge of the visual system, although recent work on more complicated tasks, 
with respect to afterimages, also seems to gain more importance. The study of 
absolute as well as incremental thresholds leads naturally to the use of small 
signals, culminating in the important work of De Lange on critical flicker 
fusion (CFF) with modulated light. Electrophysiology, however, has been 
studied chiefly with large signals, especially flashes, even when background 


illumination (light adaptation) was used. This gives rise to large and highly 


synchronized nervous signals. 
The present study describes the use of comparable signals for psychophysi- 
cal and electrophysiological measurements. My associates and I call this the 


_ weak signal study. 


Although our own electrophysiological work deals only with slow potentials 
and is still far from complete, the signal transmission could be followed with 
regard to linearity; it has been proved that the method of linear approximation 
introduced by H. De Lange (1957, 1958) permits better comparison between 
subjective and objective measurements in man. A special feature is that, 
under favorable conditions, responses can be recorded that are below the sub- 
jective threshold. A simple discussion of the statistics of threshold flicker will 
support the hypothesis that for small incremental signals an approximately 
linear character is maintained to a later stage in the process. 

This kind of treatment appears promising, but it has been shown, on the 


other hand, that the decision problem even in the simple process of flicker 


fusion is complicated and far from solved. 


Flicker Fusion 


Until 1955 the data on flicker fusion and flicker responses in the electroretino- 
gram (ERG) and electroencephalogram (EEG) were not unified by any inclusive 
theory. De Lange (1957, 1958) introduced the use of modulated light and 


_ gave us a definite analysis of this problem. By this modulation it is possible 


to change the stimulus (depth or frequency of modulation) without changing 
the adaptation of the retina. A plot of frequency versus depth of modulation 


- for flicker threshold at different intensities can be made. 


FicureE 1 reproduces one set of De Lange’s psychophysical curves, which 


=: were measured with a field of 2° with an equal surrounding for threshold vision 


* The work described in this paper was supported in part by research grants from De 


: Medisch-fysische afdeling T. N. O. van de Gesondheidsorganisatie and from the Nederlandse 
_ Organisatie voor Zuiver- 


Wetenschappelyk Onderzoek, both of The Hague. 
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at different intensities. These curves show an extremely high attenuation of 
frequencies above 15 cps. De Lange argued that at these threshold conditions 
the visual system behaves linearly with respect to the stimulus, and he proved 
also that at these frequencies the threshold is determined only by the ampli- 
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Ficure 1. Attenuation characteristics for white light of the right-hand system fovea 
brightness perception:.2° test field, with a large surrounding field having the same luminance 
as the test field, and central fixation. ‘There are four variations in the luminance as shown: 
1 and 4 are the shape of modulation, and 2 and 3 are the mode of interruption. No. 4 is ap- 
proximately sinusoidal; ripple-ratio r= fundamental B,/mean luminance B, of the stimulus; 
CFF = critical flicker frequency; ro = the critical value of the variation at the site of the 
threshold mechanism, measured at the input if no attenuation in the system occurs; the 
attenuation 7 = ro/r. Reproduced by permission from De Lange, 1957. 
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tude of the first Fourier component. This is true only if the harmonic content 
is not too high (Levinson, 1960). High frequency attenuation must occur 
before the linear transformation is disturbed. From Veringa’s work (1958) it 
can be concluded that this linear character is approximated, even at modula- 
tion percentages several times above threshold. We shall come back to this 
linearity problem later. 


Successive and Simultaneous Contrast 


Most of our own measurements have been performed by a modulated tele- 
vision projection tube as described by Denier van der Gon ef al. (1958). The 
maximal luminous fluxes (sometimes used in electrophysiological work) were 
about 1 lumen ina field of 1 steradian. They were produced by projecting the 
light source through a condenser into the pupil of the eye (Maxwellian view). 

‘In direct view, an annulus is presented of 8° inner diameter and 2° width, 
representing an apparent brilliancy of maximum 200,000 lux equivalent. For 
lower intensities, filters were used and usually the tube was run at a lower level. 
We did not enter scotopic conditions. The light output could be modulated 
according to any electrical signal up to frequencies of 100 cps, and flashing 
techniques with regulated average intensity also could be used. 

The psychophysical experiments were usually performed by presenting the 
light from the modulated source through a hole in a white screen of 20 X 20 
cm. Surrounding fields were generally made by conventional projectors with 
adjusting blue filters to compensate for the difference of color. | Usually the 
subject was at a distance of 3.5 m. and we used a pinhole pupil of 1.5 mm. 
The thresholds were determined from below and from above flicker vision. 
Under most conditions this determination could be made with great precision; 
in small fields without surround, at low intensities and at low frequencies, a 
determination takes longer, and the accuracy is less. The curves presented 
give an impression of the accuracy of our results that we consider good enough 
with respect to the conclusions and quantitative relations derived from them. 
Therefore we preferred fast determinations of a whole family of curves to a 
more accurate approach for each one separately. Most of this work has been 
described in detail by Denier van der Gon (1959). 

In FIGURE 2 a rise of threshold for large fields is shown at A, with a black 
surround below 10 cps. FicurEs 3 and 4 complete these findings also for 
smaller fields and show the relation between the thresholds with and without 
surrounding field and for different visual angles. Kelly (1959) presented 
curves in which the flickering field faded out toward the periphery, and he re- 
ported an occasional very steep fall below 15 cps. However, we could not 
reproduce his results, perhaps because our experimental conditions were not 
oe fields (> 2°) the main difference between the results with a black 
surround or an equally illuminated field is found in the frequencies below 10 
cps. We assume that in the case of equal surrounds, at low frequencies, a taal 
spatial contrast determination is made by the subject. For small fields, peach: 
olds are usually higher with a black surround than with an equal surround. 
In this case also frequencies > 10 cps are influenced. 
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Ficure 5 shows the dependency of the threshold of 12 cps flicker on the 
‘lumination of the surrounding field. A maximum sensitivity (minimum 
threshold) is found when there is no difference between surrounding and flicker 
field. However, when a black ring is introduced between the two, a minimum 
can still be found, although it is not so low. The surrounding illumination, 
however, has to be increased manifold to reach such a minimum. In this case 
there is nowhere else a field that can be used for comparison. Therefore, in our 
opinion, the lowering of the threshold here is a result of the total illumination 
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_Ficure 2, Threshold modulation depth as a function of frequency (De La 

with subjective determination, on the basis of ern A TA Rel ee 
trodes. Although the electrophysiological curves have a higher modulation percentage which 
can be ascribed to the disturbing influence of the spontaneous activity, the high-frequency 
portion is comparable to that in subjective vision whereas the low-frequency portion is totally 


rata Reproduced by permission from Acta Physiologica Scandinavica (Kamp et al., 


condition of the retina, and contrast determinations at frequencies >10 cps 
are chiefly successive contrast determinations. The influence on the sensitiv- 
ity is a function of the distance. It is improbable that these effects are pri- 
marily results of light diffusion in the eye. Within this concept it is also under- 
standable why fields of >2° at 10 cps and higher do not show much difference 
whether a black or a white surround is used. The influence of the surround 
in the case of small fields is here probably replaced by the influence of the 
illumination of the field on itself; this field is large compared with the distance 
at which this kind of influence becomes negligible. 

The dependency of the differential successive threshold on the surrounding 
field was also found for single flashes (positive and negative). 
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Our explanation differs considerably from that of Kelly (1959). 

We may conclude that at frequencies greater than 10 cps all flicker measure- 
ments are successive threshold determinations, the highest differential sensitiv- 
ity occurring with an evenly illuminated retina. At and near threshold there 
are no large deviations from linear functioning. A reduction of high frequen- 
cies occurs at an early stage. 


Simple Quantitative Considerations 


We shall see that it is possible to approach the flicker fusion problem electro- 
physiologically and even to find responses below subjective threshold. Al- 
though, owing to lack of knowledge about such fundamental data as quantal 
efficiency and integration time, a discussion can be only general, it still seems 
useful to reconsider the CFF threshold concepts and functioning in a semi- 
quantitative way. 

Since Rose (1948, 1953, 1957) and De Vries (1943), there has been much 
discussion about the optimality of the visual system with respect to physical 
limitations caused by the quantum character of the light. Most experiments 
have been performed on incremental or absolute threshold vision and, although 
often contradictory in details and interpretation, they indicate that with rea- 
sonable assumptions of quantum efficiency and summation times the function- 
ing at moderate intensities cannot suffer too much loss of information. Denier 
van der Gon (1959) studied the problem of visual acuity with artificial pupils, 
enabling a mathematical approach to the quantum distribution at the retina. 
He could describe the experimental results with the probability that in one 
summation time more quanta were falling in a light detail of, for example, a 
Landolt C, than in a dark detail. In this way his results could be translated 
into the statistics of incremental thresholds. To give a macroscopic impression 
of detectability as a function of the statistical probability, FIGURES 6a and b 
represent a Landolt C with its gap to the left, which has been constructed by 
random arrangement of the dots. The light quanta are shown as black dots; 
thus the figures can be compared directly with a Landolt C. On the average, 
three quanta fall in a light part and one in a black detail in one summation 
time. The probability that more quanta will be found in a light detail than 
in a dark one is 78 per cent in this case. This probability, given by the distri- 
bution function of the difference between two Poisson variates, can be estimated 
with help of a normal distribution and may be written as a function of the ratio 
k between the mean p and the square root o of the variance of this normal 
distribution: & = (1 — m2)/~/(m + m2). In the given case (m = 3, m2 = 1), 
an approximation can be found by uw = 2 ando = 2. If An =m — 2<n 
and m = (m + ms)/2, this formula yields k ~ An/ ~/2n. From the set as 
presented it is nearly impossible to tell where the gap would be. The figure 
composed of the 10 images, described by k = 3 (probability > 99.8) , shows that 
if no data about the possible positions and shape of the gap are given a high 
probability to exceed is needed to ascribe a pattern to such a distribu tion. We 
are quite aware that this representation might have a primarily didactic value. 
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We now reconsider FIGURE 4. At 10 cps (350 lux equivalent, pupil 1.5 mm., 
retinal illumination 200 trolands) the threshold is found at 1 per cent of modu- 
lation for a field of 22’. As we have said, under these conditions (equal sur- 


rounding field) the flicker problem may be treated as a purely successive phe- — 


nomenon. Contrary to most situations in incremental threshold studies, the 
surfaces to be compared may be assumed to be equal in this case. A simple 
assumption would be that for CFF the criterion will be the probability that 
the positive part of the sine wave contains more quanta than the dark (nega- 
tive) part. Since at 10 cps the system seems to achieve its highest sensitivity, 
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Ficure 5. Critical modulation depth dependent on the surround: the frequency is 12 
cps; the brilliancy of the test field is 60 lux eq.; and the artificial pupil, 1.5 mm., is used. Curve 
I represents the surround without interruption and Curve II is the surround interrupted by a 
black annulus of 24min. The sensitivity (reverse of the modulation percentage for the thresh- 
old) reaches a maximum in both cases but, when a black annulus is introduced, shifts to a 
much higher value of illumination of the surround. Adapted from Denier van der Gon, 1959. 


a summation time of 50 msec. is chosen, differing not much from the conven- 
tional assumptions at these illumination levels. With an over-all (foveal) 


quantum efficiency of 1 per cent for the white light used (5500° eq.), the average — 


number of quanta effectively absorbed will be VN = 4.2 & 10 quanta/50 msec. 
The net difference between two sine-halves at 1 per cent modulation depth for 


threshold will then be D = 5.4 X 10? quanta = 1.85 ~/2N, which yields a | 


probability of finding more quanta in a positive than in a negative sine-half 
of ~97 per cent (See FIGURE 6b). I emphasize at this point that the eye prob- 
ably has no special constructions for detecting temporal periodicity and that 
the problem therefore can be treated on the chosen statistical basis. The high- 
est estimate for quantum efficiency for white light might amount to 4 per cent. 
We chose a conservative value, however. A higher efficiency would not change 
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essentially our discussion, which mainly serves to determine whether optimality 
considerations might also be useful here. Considering also the possibility of 
the influence of a memory, the result is a reasonably safe device for detection. 

We do not think it fruitful to present a model of the way in which the tem- 
poral comparison is done. An interesting phenomenon, possibly connected 
with this process, is that subjective discrimination of the frequency of flicker is 
possible only up to about 18 cps, if adjusted in the depth of modulation with 
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istributi i (reversed). In a light de- 

: antum distribution at the retina for Landolt C (rey : tr 
at mein, ee the gap), an average of three quanta are distributed in 145 sec., aes 
pared with one in a dark detail. Ten exposures of 145 sec. each are presented in (q). he 


_ probability of finding more quanta in a light detail than in a dark one is 78 per cent. Even 


ats : eet a - 
i ted, it is extremely difficult to find the direction of the gap. 
ae tat ctl dificult, to find the gap (directed to the left) in the figure (b) composed 
= ed b 


~ of all 10 presentations. Here the probability to exceed is 99.8 per cent. If this kind of pres- 


i i ision, it shows, as Rose did for contrast, 

ion is allowed for comparison with threshold vision, it s a se ¢ si 

Pie octability to Leecd is needed to perform a judgment, Adapted from Denier 
van der Gon, 1959. 
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regard to the CFF curves. At higher frequencies the discrimination fails; 
the low subjective frequency of flickering fluorescent tubes is well known. It is 
tempting, therefore, to assume that the summation differs from the normal 
integration caused by the high frequency attenuation in early parts. 


“On” and “Off” Concepts 


We shall restrict this discussion to the threshold problem, since the impor- 
tance of “on” and “off? elements in man for important biological functions, 
although not explicitly shown, could easily be accepted. 

The importance of these elements for visible acuity and contrast phenomena 
is sometimes explicitly stated (Ditchburn, 1959; Kelly, 1959), and sometimes 
implicitly, but it is not quantitatively argued (Le Grand, 1956). The gener- 
alized idea is that by movement of the eye, spatial gradients are trans- 
duced in temporal events. Therefore a comparison with flicker, where all 
events are temporal, is interesting. Doubt about it can be found in the work 
of Denier van der Gon (1959) and, very recently, in that of Hebbard and Marg 
(1960). The importance of the micronystagmus is sometimes argued, con- 
sidering the results obtained with stopped images, in which case the visual 
impression tends to disappear. The disappearance time of the image, how- 
ever, is far greater than the time needed for reaching a high visual acuity. 
Moreover, it is obvious that, in an optical system, moving of the image impairs 
detectability except when information about the position is available. This 
is the case only for larger saccadic movements. That the latter have a func- 
tion for the maintenance of vision, however, seems very reasonable.* 

The stress usually laid on the on and off elements is a natural consequence 
of the common experimental technique in electrophysiology, using large and 
sudden signals. In the case of modulated flicker, an influence of on and off 
elements must be considered in a generalized way, since we deal only with more 
or less slowly varying stimuli. In fact, every successive contrast decision is 
an on or off decision. This applies equally to flicker and to single flashes. 
It is not easy to extract from the literature what times are involved in estab- 
lishing an on or off effect, most results having been gathered with rather large 
transients. To make a sensible distinction with the usually accepted general 
summation time of the visual system of 50 msec. we have chosen a “differentia- | 
tion” time of 10 msec. This is probably not too long, compared with the _ 
micronystagmus frequencies presented in the literature, which are quoted up— | 
to 200 cps (when using conventional calculus, 10 msec. would be equivalent to 
100/ 2m = 16 cps). The successive judgment described earlier with a summa- 
tion time of 50 msec. was performed according to a difference of 5.4 & 102 
quanta. If the comparison must be done each 10 msec., according to our as- 
sumption of on and off times, the statistics will be impaired by about V5 ata 
frequency of 10 cps. We see that from k ~ 1.8, we reach k ~ 0.8, which 
means a heavy loss of discriminability. Even when a higher quantum effi- 
ciency had been assumed (compare with FIGURE 6), this would be important. 
(If this simple model for successive comparison could be used, it is easily seen 


* After this manuscript was completed m i i soul 
: 3 y attention was directed t Ri 
(1958), in which a very clear review of this problem can be found. oa Pa 
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that at low and at high frequencies attenuation would occur. At high fre- 
quencies, however, “resonance” phenomena would be expected, but could not 
be found in our experiments with white light.) 

From this we may conclude that psychophysical and elementary quantum 
considerations, to be supported by our electrophysiological results, make it 
plausible that the flicker threshold judgment is done at a rather late stage 
(Rushton, 1953; Brindley, 1960). After the decision has been made, the linear 
relation between signal strength (modulation depth) and response is taken 
over by a nonlinear relation. An early place for this transformation is not 
easily brought into accordance with our finding of subthreshold responses in 
cortical derivations and our other results. In our simplified theory on and off 
effects would certainly not improve threshold vision. Their use might be 
found in editing important and larger signals in a safe and economical way. 


Logarithmic System 


Generally the visual system is considered to exhibit logarithmic properties. 
Of course this logarithmization is only an approximation, as are nearly all 
mathematical formulations of physiological phenomena. Certainly in the non- 
linear transformation two features may be distinguished. We call these the 
immediate logarithmization and the slower adaptational process. Data on the 


- pupillary reflex of man, that correct earlier material, show also that for initial 


steps, over a large range, an approximate logarithmic functioning occurs. 
FicuRE 7 shows the pupil contraction as a function of the intensity of the light 
step. As may be expected with logarithmic functioning, at low intensities 
deviations must occur and, at high intensities, some extra saturation is present. 

After a longer time the light adaptation diminishes the sensitivity of the 


4 (primary?) system still further, without too great an increase of the statistical 


fluctuations, however. ; ; 
Furthermore, because of the use of suprathreshold modulations in our elec- 
trophysiological experiments, we shall discuss some single properties of logarith- 
mic systems, abstracting from actual processes. 
The finding of a more or less linear transmission of increments can be brought 


- into reasonable accordance with logarithmic function. The development of 
~ log I (1 + Mcos wl) yields R = log J + Mcos wt — (M?/2) cos’ wt + (M*/3) 


cos*wi — ---. J isarelative measure of the intensity and, in our case, is sup- 


- posed to be much larger than 1. For modulation depths of 25 per cent the 
amplitude of the second harmonic (M?/4 cos 2 wi) is only 146 of the amplitude 


of the fundamental in the response, and the average shifts from log J to log J — 
M?/4 = log I — 0.015. 


M?/2 cos? wt = M2/2 (1 — cos 2 wt)/2 = M?/4. 


_ Furthermore, a property of a logarithmic system is that the response to a con- 


stant depth of modulation is independent of the intensity. The depth of modu- 
lation of the response will be, in our approximation, the M/log I for small 


~ modulations. Nevertheless, at high intensities, where the depth of modulation 
— of the response is smaller, the subjective discrimination is better; the approxi- 


mate modulation threshold in a considerable range is M ~ 1/ /f. It seems 
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also that the threshold setting is done by the quantum fluctuation itself or by a 
kind of learning, by which the visual system has organized itself not far from 
optimal (Brindley, of. cit., p. 196, discusses this question). 
The problem of linearity as discussed by De Lange with respect to Talbot S 
law can also be treated in the same way. Talbot’s law can be expressed in the 
simplest way by the following. Light seen as stationary has the same intensity 
as the physical average. Thus in FIGURE 4 we see that at 50 cps the CFF will 
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FicurE 7. Pupil contraction as a function of flux, compared with data of other authors. — 
In the present material there is a clear distinction between curves for maximum contraction 
and for contraction measured after a certain time (rebound). Reproduced by permission 


from Acta Physiologica et Pharmacologica Neerlandica (Van der Tweel and Denier van der 
Gon, 1959). 


be at 200 per cent modulation (short flashes). The threshold at very low 
frequencies is here 1.5 per cent, which may be considered a real simultaneous _ 
threshold at these low frequencies. Log 1.015 J ~ log J + 0.015. This is 
the same difference as that found for the shift of the average for a depth of 
modulation of 25 per cent. From our simple formula we see now that if at 
50 cps an eightfold attenuation occurs, the depth of modulation of 25 per cent — 
remaining from the original flashes would result in a shift of the average level 
equivalent to a 1.5 per cent shift of the original intensity, also just at the level 
of detectability. If Talbot’s law is not discarded yet, this proves that there 
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is enough high-frequency attenuation before nonlinear processes play a role, 
but this attenuation need not be more than eightfold at 50 cps while, sub- 
jectively, according to our curves, the sensitivity has decreased about 150 times. 


Electrophysiological Experiments 


With different methods of analysis and of recording, electrical responses to 
modulated light and sudden changes in illumination have been studied (Van 
der Tweel et al., 1958; Kamp et al., 1960; Van der Tweel and Visser, 1959; Visser 
et al., 1960). Electroretinograms (ERGs) were recorded in cat and man with 
simple averaging and also with sensitive integration (integrator of A. Rémond). 
As many as 2000 responses have been used for integration at high frequencies. 
Occasionally experiments were done on the lateral geniculate body of the cat 
and with intracortical electrodes in man, all with selective amplification. A 
number of integrated curves have been recorded with normal scalp electrodes. 
Some of our results most relevant to the present discussion will be reported. 

The fields used were always rather large, not only to get large responses but 
also to decrease eventual influences of stray light. The retinal illumination 
used in man was not exactly known. In addition to other instrumental prob- 
lems the contact glasses made it difficult to obtain an even illumination of the 
retina. The losses in the contact glasses were not always the same, and the 
natural pupil had to be used because a continuous use of mydriasis produced 
too great a strain. This may account for a certain variability in the response 
curves. No differences were found, whether the room was moderately illumi- 
nated or not. Certainly improvement of the optics would be useful and, with 
the development of new techniques, it may become possible to measure re- 
sponses with smaller fields in conditions comparable to foveal flicker experi- 
ments. The subjective curves measured under identical conditions, as in our 
electrophysiological experiments, however, show no important deviations from 
those obtained in smaller fields. Only in two cases was mydriasis used to see 
whether phase shifts found at low frequencies could be attributed to the pupil- 
lary reflex. We found no difference that could not be attributed to differences 
in luminous flux. When the nonrecorded eye was stimulated no “responses” 
were found even with a great number of integrations. 5 

~The retinal responses with fields of 1 steradian and illumination levels of 
about 200 to 100,000 trolands were approximately sinusoidal in cat (FIGURE 8) 
and in man at depths of modulation of 25 per cent or less. Although responses 
of less than 10 pv are easily disturbed, especially at low frequencies, by noise, 
the amplitude and phase characteristics were determined with reasonable ac- 
curacy. They are shown for one human subject in FIGURES 9a and 6. Some 
facts seem important. To a first approximation, the ERG behaves linearly at 
these illumination levels and with modulation percentages up to 25 per cent. 
This is also proved by the recording of responses to positive and negative step 
functions. These show a fair correspondence with the responses calculated 
from the phase and amplitude characteristics (FIGURE 10a and b) despite the 
irritability of the eye in these long sessions. At 25 per cent modulation there 
are, however, in addition, nonlinearities, as shown by FIGURE 11, in a re- 
sponses to positive and negative incremental steps are shown, At 12.5 per 
cent this seems to have improved, but we could not determine whether all 
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nonlinearities would disappear; with smaller steps the noise became relatively 
too large. The time constant of 0.7 sec. did not permit us to follow the step 
response longer than about 300 msec. Recent measurements by Schweitzer 
and Troelstra in the Institute for Perception, Soesterberg,* with DC recording 
make it probable that the predicted long-lasting zero shift indeed occurs, al- 
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20uV 
| 25% 25% 40% | 
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Ficure 8. A set of frequency responses in the electroretinogram (ERG) of the anesthe- 
tized cat (pentobarbital): the retinal illumination was about 50,000 trolands and the field 
~56°. The upper trace of each pair represents the ERG higher frequencies with super- 
position. The lower trace represents the AC output of the photocell controlling the light 
source. The phase shift at 3 cps is already 180°. At 5 cps, the amplitude is lower than 
at 3 cps and, at 6 cps, a distortion is clearly visible. At 12 cps the modulation had to be in- 
creased. ‘The “normal” ERG is recorded with a step from dark to light. The last vertical 
row shows incremental step functions. The responses to positive and negative increments 
are reasonably symmetrical except perhaps for the first rapid deflection. Reproduced by 


permission from the Proceedings of the International Symposium on Electroretinography (Van 
der Tweel and Visser, 1959). 


though here also our data do not permit a definite conclusion about the sym- 
metry for positive and negative increments. 

It is difficult to choose certain features from the phase or amplitude charac- 
teristics or, even better, from the R cos ¢g curves, that could correspond to 
physiologically distinct processes (these curves, constructed from the amplitude 
R and the phase ¢, were used to calculate the step response). The same ap- 
plies to the conventional way of distinguishing characteristic waves in step or 
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impulse responses, which are also totally descriptive for the system in the 
linear case. It is not clear which combination of functions is incorporated 
in the appreciable phase shift at low frequencies. It is tempting to distinguish 
at least a slow and a fast process, perhaps in the conventional way, since the 
phase and amplitude characteristics cannot be described by a minimum phase 
shift network or by a simple delay. 

The amplitude relation between low and high frequencies changes with the 
intensity. Normally the higher frequencies at high intensities are relatively 
enhanced. At a frequency of about 10 cps the response is nearly intensity- 
independent and, on the whole, the intensity is not found to be an important 
factor in the amplitude of the response below 15 cps. This supports the con- 
ventional assumption of a more or less logarithmic function. At high intensi- 
ties, the attenuation of high frequencies is much less than in subjective vision. 
The results in these frequency regions show a high reproducibility. 

It may be significant that at higher frequencies subthreshold retinal responses 
could be recorded. With 2000 integrations, a response could be recorded at 
80 cps and 25 per cent depth of modulation, as shown in FIGURE 12. With 
so many integrations one always must be careful with respect to eventual 
photoelectric effects. The dependency of the phase on the frequency makes it 
improbable, however, that the response was not due to retinal activity. Ina 
number of experiments that we succeeded in recording at low depths of modu- 


— lation but above threshold, the responses were still approximately proportional 


to the modulation. It will be most interesting in these cases to see whether 
distinct alterations will appear on passing the subjective threshold. With re- 
gard to this it is important to remember that the fields used were large com- 
pared to the integrative properties for subjective vision. 


Lateral Geniculate Body of Curarized Cat 


These experiments were meant. only as preliminary ones to see whether 
modulated light could also be used in these regions of the optical pathways. 
Electrodes were brought into the lateral geniculate body (1.g.b.) of curarized 
cat (operated under local anesthesia) by the Horsley-Clarke technique. For 


most of this work a high illumination of about 50,000 trolands and a field of 


15° was used. We present some of the results here because they indicate that 


in the l.g.b. a certain linearity is maintained, and we learned something about 
_ harmonic distortion. 


There was a great variety of responses but, at a number of places, using 
selective amplification (Q = 20), response curves were measured. The accu- 


_ racy of the curves is not very high because of the inherent noise that, however, 


when measured with an averaging rectifier, was not too dependent on frequency. 
Ficure 13 gives some of our results. It is seen that there is attenuation at 
the low and at the high-frequency side. Modulations down to 0.5 per cent 


~ could be traced in the response. A high second harmonic content was found, 
~ much higher than could be accounted for by ascribing the distortion to a loga- 
- rithmic function. The harmonic content was increased at higher modulation 


percentages. At some places a linear relation with the modulation was found 


3 up to a certain percentage. In one case, at 22 cps, with modulation of more 
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than 20 per cent, the responses at the proper frequency disappeared, but at the 
same time 44 cps was found with a large amplitude. The response at these 
frequencies in this well-controlled experiment is presented in FIGURE 14. At 
some places below 5 cps no slow responses were found, but in the loudspeaker 
deep modulations of the light produced a hissing rhythmic sound (frequency- 
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Ficure 9(a), Amplitude (a, above) and ph ite) i i 
mpl h phase response (b, opposite) in the electroret 
of man, recorded with integrating technique: visual field Og Sapettn pap: and raaxteand 


brilliancy ~20,000 lux eq. Note the relativel i i i 

: L ; \ y low attenuation at high frequencies com d 

ve oe 3 and 4. The amplitudes do not differ substantially: for fie two intensities 
€ amplitude accuracy below 3 cps is not very high. The considerable phase shift at low 


frequencies b ill hi i iti i i i 
= anene 7 eons still higher at low intensities. On the amplitude scale 1.0 is equivalent 


modulated action potentials). Apparently there were at least two representa- 
tions of the modulated signal. From a recent experiment with lower intensities 
(about 200 lux, diffuse on the eye) in several points of the optical radiation 
and l.g.b., confirmation of our previous results was obtained. Ficure 15 
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shows responses at different modulations for the fundamental and the second 
harmonic. At 10 per cent the fundamental is larger than at 50 per cent, while 
the latter evokes greater second harmonics. The important results show that 
large signals are not always the best means of evoking large potentials and that 
complicated interactions take place when the stimulus exceeds certain levels. 
Of course these results do not suggest that this is the only way in which the 
system operates. 


Objective Recordings in Man 


This part of the work has been done in collaboration with several institutes. 
A patient of Sem-Jacobsen’s who had some of the electrodes implanted in an 
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Ficure 9(6). See legend on the opposite page. 


optically sensitive primary part of the cortex was the subject. The patient 
was considered for leucotomy, and the depth electrode technique was used to 
restrict the area to be excised (Sem-Jacobsen et al., 1956). We used a modu- 
lated television tube apparatus. The light was projected in the conventional 


i il was used. Normally a 
to the eyes of the patient. The natural pupil was use 
; cs of 500 to 50,000 lux eq. was used as a brilliancy of the presented annulus. 
- Although we had to use the technique of selective amplification and, at this 


s =} (a3 ”? “| 
stage of our research, still tried to measure De Lange” curves with reference 


to the noise, we present these results also because we think it was rather im- 
_ portant that, with modulated light, responses have been recorded intracortically 
: P) 
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in a fully conscious man. The detailed technique and results have been de- 
scribed elsewhere, but some of these results bear on the questions treated in 


this survey. 
FicurE 16 shows the selectively amplified signals of three different intra- 


rel.ampl. 


ERG man 


no filter 


rel.ampl. 


filter 1/64 
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Ficure 12. Subthreshold response at high frequency under the same conditions as 
those in FIGURES 10 and 11. 
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Ficure 13. Bipolar recording in lateral geniculate body (1.g.b.) of curarized cat, using 
gross electrodes, modulated light at 15°, and 50,000 trolands. ‘The curves represent the mod- 
ulation needed for a constant response at the fundamental frequency of the modulation and 


for the second harmonic. Recorded with selective amplification (Q = 20). At low and high 
frequencies a steep attenuation is shown. 
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cortical leads. The electrodes were about 1 mm. long and 50 y thick and sep- 
arated by about 10 mm. It is seen that with 50 per cent modulation the re- 
sponses of the second harmonic and those at the fundamental (10 cps) do not 
reach their maximum at the same time and at the same place. In FIGURE 17 
an example is given of the response at 10 cps with a modulation depth of only 
5 per cent. The CFF curves show a high attenuation below 8 cps and also 
above 25 cps. Objective curves of this patient and subjective of the author 
under comparable visual conditions are presented in the same FIGURE 2. Al- 
though the slopes will be a function of the noise spectrum, the steep fall at high 
and low frequencies cannot be expected to undergo large corrections for constant 
response curves. The responses were not very dependent on the intensities 
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Ficure 17. A clear response is shown in channel 1 with a modulation depth of only 5 
per cent. The modulation happened not to be decreased instantly. The amplitude of the 
analyzed signal stays constant for a short time after the decrease in modulation depth. It 
is not improbable that this is still a real response. Reproduced by permission from Acta 
Physiologica Scandinavica (Kamp et al., 1960). 


used within a 25-fold range. Here also second harmonic responses could be 
found that were stronger than those expected according to a logarithmic law. 

A number of experiments was also performed on two normal human subjects 
with integrating technique and scalp electrodes. FIGURE 18 shows a response 
at 44 cps, definitely below threshold, compared with the “noise” in the same 
channel. FicurE 19 gives a response at 5 cps and 25 per cent modulation. 
The doubling of frequency shows that here, too, flicker processes exhibit a com- 


plicated structure, which has not been accounted for in our theoretical consid- 
erations. 


Discussion of the Electrophysiological Material 


If we accept the l.g-b. curves of the cat to have some importance also for 
man, we see that whereas in the ERG the high frequencies are not nearly as 
much attenuated as in perception, this is more the case in the l.g.b. and also 
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in responses from certain cortical regions in man. A high sensitivity in l.g.b. 
and cortex is found at about 10 cps. The responses are of a slow kind because 
they influence the selective amplifiers, which would not be the case if they were 
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ing 1 i i Below CFF, 1000 
: Ip electrode recording in man with large visual field. l ' 
ee Eins : on ee are represented. There is a clear response. The circles show the 
Mena given when light was interrupted. Channel 7 of the integrator was apparently mal- 
adjusted. 


composed of action potentials. They may represent reasonably well the a 
oidal input signal as does the ERG. From the latter it is clear ps the phase 
shifts correspond with certain features of the step Tesponse, which are abies 
patible with a minimum phase shift network. It might be useful to wre 
incremental step responses from frequency response also in cortex and l.g.b. 
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Second harmonic responses found at comparatively low modulation per- 
centages (but until now not below subjective threshold and tending to decrease 
with the modulation depth) show that in cat and man also essential nonlinear 
functioning occurs. The picture of flicker fusion is complicated by the fact 
that the amplitude characteristics at different points within the same anatomi- 
cal region may differ substantially, especially below 5 cps, at which frequencies 
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Ficure 19, The recording conditions are similar to those used i i 
_ Fic Pape n sed in FIGURE 18, but there is 
flickering subjectively. At 5 cps there is doubling of the presented frequency. ; : 


the human eye is still very receptive to flicker. In a number of cases, only 
very low responses have been found in these frequency regions; with intepnae 
tion, however, some responses at 3 cps were found with scalp electrodes. 

We have presented these incomplete results to show the possibilities and 
difficulties of the method. A summary may be presented: 

Sinusoidal modulation of light is represented in different parts of the visual 
system. The transmission is more or less linear and is recorded in some cases 
even below threshold. The amplitude function of the ERG differs in many — 
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aspects from the subjective one. Some l.g.b. and some intracortical curves 
have large attenuations at low frequencies. The amplitudes or ‘objective 
thresholds” are not much dependent on the intensity, which could be explained 
by logarithmic functioning. Nonlinearities although present are not prominent 
in the ERG, but in the l.g.b. and cortex they are certainly stronger than can 
be explained by a logarithmic function. The measured representations are not 
the action potentials themselves but apparently a not-too-distorted image of 
their frequency. \ i 

The complicated character of low-frequency responses Jeaves the question 
open as to whether flicker is represented in more than one way and also which 
responses reported in this work could be the adequate ones for flicker perception. 

{ 

Final Comment \ } 

We are well aware that our treatment and part of our experiments are still 
rather crude; however, they offer a basis for discussion. Certainly important 
changes in our concepts will be needed when more material is available. We 
hope, however, to have shown that the method of linear approximation in 
electrophysiology may lead to a better understanding of flicker phenomena. 
The distinction of strong and weak signal experiments might have value also 
for other fields of physiology. Frequency modulation of repetitive stimuli 


- might then take the place of the modulation of light in our experiments. It 


would seem that further analysis of electrophysiological responses in man 
would help to resolve the important problem of how quantum statistics’ in- 
fluence our threshold decisions. 


Acknowledgments 


It is nearly impossible to mention everyone who has been of importance in 
preparing this manuscript. Although of course the responsibility remains with 
me, the whole work is a direct consequence of the ‘‘school” of biophysics started 
by the late H. L. De Vries in Holland. Discussions in the “sense physiology”’ 
group of the Stichting Biophysica, Utrecht, greatly influenced my work through 
the kind but critical analysis of problems that prevails there. I am especially 
grateful to M. A. Bouman, H. De Lange, and A. Vendrik. As is clear from 
the quoted literature, the collaboration with J. J. Denier van der Gon has been 
of the greatest value to me. J. Strackee also contributed much to the final 
result. 

Most of the electrophysiological work was done with P. Visser. J. ten 
Cate gave us hospitality for our experiments. A. Rémond’s great knowledge 


and technical skill were of great importance in the use of his integrator for 


parts of this work. C. W. Sem-Jacobsen and W. Storm van Leeuwen were 
especially helpful in the Oslo experiments. F. T. H. Veringa’s critical dis- 
cussions also are appreciated very much. As they have in the past, H. den 


Hartog and F. A. Muller devoted much time to my problems. . 
Ficures 9, 10, 11, 12, 18, and 19 were recorded together with A. Rémond 


4 (Laboratoire EEG, Salpétriére, Paris, F rance) and Visser (Laboratory of 
_ Physiology, University of Amsterdam, the Netherlands). Ficurrs 13 and 14 
employ data derived by Van der Tweel et al., 1958, and ricurE 15 was recorded 


856 Annals New York Academy of Sciences 


with K. Mechelse and M. van Vliet (Neurology Clinics, University of Leiden, 
Leiden, the Netherlands) and Visser. 


References 


Brinviey, G. S. 1960. Physiology of the Retina and Visual Pathway. Edward Arnold 
(Publishers) Ltd. London, England. : es 

Dr Lance, H. 1957. Attenuation characteristics and phase shift characteristics of the 
human fovea-cortex systems in relation to flicker fusion phenomena. Thesis. Delft 
Univ. Delft, Netherlands. 

Dr Lancer, H. 1958. Research into the dynamic nature of the human fovea cortex sys- 
tems with intermittent and modulated light. I and II. J. Opt. Soc. Am. 48: 777-789. 

DENIER VAN DER Gon, J. J. 1959. Gezichtsscherpte, een fysisch-fysiologische studie. 
Thesis. Univ. Amsterdam. Amsterdam, Netherlands. 

DENIER VAN DER Gon, J. J., J. STRACKEE & L. H. VAN DER TWEEL. 1958. A source for 
modulated light. Phys. Med. Biol. 3: 164-173. 

De Vries, H. L. 1943. The quantum character of light and its bearing upon the threshold 
of vision, the differential sensitivity, and acuity of the eye. Physica. 10: 553. ¥ 
Dircupurn, R. W. 1959. Physical methods applied to the study of visual perception. 

Bull. Inst. Phys. : 121-125. 

HEBBARD, F. W. & E. Marc. 1960. Physiologic nystagmus in the cat. J. Opt. Soc. Am. 
50: 151-155. 

Kamp, A., C. W. Sem-JAcossEn, W. StorM VAN LEEUWEN & L. H. VANDER TWEEL. 1960. 
Cortical responses to modulated light in the human subject. Acta physiol. scand. 48: 
1-12. 

ane D.H. 1959. Effects of sharp edges in a flickering field. J. Opt. Soc. Am. 49: 730- 

32 


Le Granp, Y. 1956. Optique Physiologique. Vol. III. : 173. Editions de la “Revue 
d’optique.” Paris, France. 

Levinson, J. 1960. Fusion of complex flicker. II. Science. 131: 1438-1440. 

Rices, L. A. 1958. Visual effects of minimal eye movements. Proc. 2nd Symposium on 
Physiological Psychology. ACR 30. Office of Naval Research. Washington, D. C. 

RosgE, A. 1948. Sensitivity performance of the eye on an absolute scale. J. Opt. Soc. Am. 


Rose, A. 1953. Quantum and noise limitations of the visual process. J. Opt. Soc. Am. 


Rosr, A. 1957. Quantum effects in human vision. Jn Advancesin Biological and Medical 
Physics. Vol. V. : 211. Academic Press. New York, N. Y. 

Rusuton, W. A. H. 1953. Aspects of Retinal Physiology. Instituto de Biofisica, Uni- 

. versidade do Brasil. Rio de Janeiro, Brasil. 

SEM-JACOBSEN, C. W., M. C. PETERSEN, H. W. Dopncg, Jr., J. A. LaAzarte & C. B. Hotman. 
1956. Electroencephalographic rhythms from the depth of the parietal, occipital, and 
temporal lobesin man. Electroencephalog. and Clin. Neurophysiol. 8: 263-278. 

VAN DER TWEEL, L. H. & J. J. DENIER VAN DER GON. 1959. The light reflex of the normal 
pupil of man. Acta physiol. pharmacol. neerl. 8: 52-88. 

VAN DER TWEEL, L. H., C. W. SemM-Jacospsen, A. Kamp, W. Storm vAN LEEUWEN & F. T, 
H. VeRINGA. 1958. Objective determination of response to modulated light. Acta 
physiol. pharmacol. neerl. 7: 528. 

VAN DER TWEEL, L. H. & P. Visser. Electrical responses of the retina to modulated light. 

‘; Proc. te ee ree Luhacovitée. In press. 

ERINGA, F. T. H. ; some properties of nonthreshold flicker. J. : 4 ; 
rhe panes prope d flicker. J. Opt. Soc. Am 

Visser, P., A. Rémonp & L. H. VAN DER TWEEL. 1960. Recherches électrophysiologiques 
= Sig eats par une lumiére modulée chez l’Homme et chez le Chat. J. Physiol. 

aris). 62: 244. 


BSh ii ial 


aN i ee 


A Noah Ree oe Tio hem iaat Nie a bh A ob Rani SO 


Part IV. The Reticular Formation 


ON CIRCUIT PATTERNS OF THE BRAIN STEM RETICULAR CORE 


M. E. Scheibel and A. B. Scheibel 


Depariments of Psychiatry and Anatomy, University of California Medical Center, 
Los Angeles, Calif., and the Veterans Administration H ospital, ‘Brentwood, Calif. 


Interaction of the biological organism with its environment depends on a 
steady flow of reasonably valid information about that environment, continu- 
ous comparative evaluations of that information in past and present reference 
frames, and the selection of suitable behavior modes. Reception, transmis- 
sion, and meaningful registration of information are now known to constitute 
problems in theoretical and practical neurology undreamt of 20 years ago. 
The last two decades have shown conclusively that the presence of sophisti- 
cated receptors, adequate transmission systems, and suitable cortical arrays 
for reception and manipulation of the data are not sufficient. 

Three concepts in particular have enriched our understanding of the way 
the brain works. The first of these was epitomized by Dusser de Barenne 
and McCulloch’s formulation of continuous two-way circulation between thala- 
mus and cortex as the basis for upper brain activity (1941). The second was 
the discovery of a nonspecific, extralemniscal system projecting widely upon 
the cortex in parallel with specific sensory paths suggested by Forbes and 
Morison (1939) and shown clearly in the experiments of Moruzzi and Magoun 
(1949) and others a decade later. The third was the interpretation of Claire 
and Bishop (1955) that dendrite shafts may function characteristically as 
decremental, nonpropagated conductors, thereby freeing them from the func- 
tional rigidity of all-or-none digital repeaters. 

Although we still know virtually nothing of the biomechanics of human de- 
cision, we feel intuitively that, for the most part, decision is not likely to be 
the product of sudden flip-flop phenomena, as when relays open or close. 
Rather, it must be a progressively formulating ‘focussing down” from a 
larger series of possibilities to a smaller one, utilizing possibilities that are al- 
ready to some extent individually weighted by the intrinsic structure of the 
situation and by the character structure and previous life experience of the 
individual involved. As the various possibilities are sifted, continuously fed- 
back loops from both inside and outside of the organism may be visualized as 


adding emphasis and credibility to some and detracting from others. Such 


shaping interactions should occur at every synaptic station, so that the final 
output as it is achieved tends to resemble the stepwise emergence of a marble 
statue from the crude block rather than the sudden appearance of typed figures 
on an output tape. 

If this analogy has validity, we must look for a system—or a system of 
systems—that is represented at every synaptic level along input and output 
channels and that, in turn, is continuously supplied with the widest possible 
spectrum of data from within and without the organism, 
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Accordingly, we concern ourselves with the nature and circuit arrangements 
of a system that appears to be arranged in parallel with the classical sensory 
channels, the brain stem reticular core. Much has been written about the 
connections and functions of this core (Henry Ford Hospital Symposium, 
1958) and much remains to be learned. Time will allow us only to outline 
something of the internal organization of this system, and its possible role in 
the neural scheme. 

The reticular core consists of a phylogenetically primitive (Kappers et al., 
1960) matrix of nerve cells, axons, and glia, extending from the lower end of 
the medulla oblongata to the anterior end of the brain stem, and then by means 
of as-yet-poorly-defined projections to paleo- and neocortex. Although this 
system does not show the highly organized architectonic stereotypes charac- 
teristic of cortical tissue, patterns of anatomical and functional organization are 
clearly definable (Scheibel e¢ a/., 1955). 


Input Arrangements 


When a typical cross section of the lower brain stem is viewed, as revealed 
by the silver impregnation methods of Golgi (Ramon y Cajal, 1952), dendrite 
processes of reticular cells are seen to project in all directions, interdigitating 
massively, and even projecting, in some cases, into specific cranial nerve nuclei 
and fiber bundles. The extension of individual dendrites is appreciable, often 
approximating one half the cross-sectional area of the brain stem in mouse, rat, 
and kitten (Scheibel, 1951), and raises questions as to the ultimate significance 
of fine nuclear subdivisions such as suggested by Olszewski and Baxter (1954) 
and others. 

When the well-impregnated sagittal section is studied, a geographical theme 
is immediately obvious. Dendrite shafts are seen to be arranged primarily 
perpendicular to the long axis of the brain stem, with a secondary and much 
less extensive system projecting rostrocaudally (Scheibel and Scheibel, 1958). 
This orientation mimics the two main directions in which axons course through 
the system. The rostrocaudal axis is followed by all efferent reticular axons 
that, as we shall see, run to higher and lower stations in the neuraxis, effecting 
continuous en passage contacts with reticular dendrites. In addition, most 
longitudinal fiber systems of the brain, both afferent and efferent, follow a 
course roughly parallel to this axis. The great majority of these systems also 
make contact with reticular cells, either by collaterals or through terminals 
that turn at approximately right angles to the parent fibers, penetrate varying 
distances into the reticular core, and establish multiple synaptic contacts with 
reticular dendrites and cell bodies. Thus the two main orientations of reticu- 
lar dendrites mirror the orientation of adjacent axonal components, and make 
for parallel axon-dendrite appositions, allowing for repetitive axodendritic 
synapses. 

If the patterns of ascending and descending fibers projecting as collaterals 
or terminals upon the reticular core are studied, extensive degrees of overlap 
are seen. Similarly, if one studies the details of the synaptic scale upon the 
individual cell (Lorente de N6, 1938), the degree of convergence of heterog- 
enous afferents upon the single unit is impressive (Scheibel et al., 1955). 
Histological and microelectrophysiological analyses show that presynaptic 
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termini from as many as one-half dozen or more separate fiber systems may 
contribute to the terminal array on any one cell. Similarly, the array on each 
cell tends to differ somewhat from the next. Thus each cell becomes an in- 
tegrating subcenter with its own unique properties, and the whole reticular 
core becomes a mosaic of units whose individual outputs can be arranged 
along a spectrum of possibilities. 

Thus far we have discussed only the direct transsynaptic communication of 
afferent impulses upon the reticular neuron. However, in the immediate 


neighborhood of each reticular soma are hundreds of dendrite shafts of near 


and distant cells. Processes of the former may be quite large, of the order 
of 2 to 5 yw, as they skirt the reticular element in question. Processes from 
more distant units may be finer than 0.5 uw in size. In addition to these den- 
drite processes passing near the soma, there may be tens of thousands of axons 
in’ the vicinity, some myelinated, some unmyelinated, but all presumably 
carrying impulses at one time or another. In the case of the axons, most are 
probably node-bearing, and it is now quite generally accepted that in such 
fibers, transmission is saltatory (Tasaki, 1953). Our individual reticular unit 
is thus surrounded by. a dense matrix of electrically active tissue elements; 
some are the site of decrementing fields collapsing along their surfaces and 
others support propagated disturbances leaping along their surfaces several 
hundred yp at a time from node to node, with ion flow lines presumably extend- 
ing considerably beyond the axon surface. Neurophysiological opinion re- 
mains divided on the possibility for cross talk among such units in the non- 
pathological state. However, there is sufficient evidence in the literature to 
indicate that, at least under some conditions, nonsynaptic (ephaptic) interac- 
tion can be demonstrated (Arvanitaki and Fessard, 1936; Arvanitaki, 1942; 
and others). Mountcastle (1957) and Mountcastle et al. (1957) have esti- 
mated the diameter of fields surrounding active cortical cells as of the order 
of 100 » or more, and assume that “‘.. . it seems possible that an active corti- 
cal neuron will influence its neighbors by ephaptic interaction.” 

In so far as we know, similar measurements have not been made for reticular 
neurons. However, in a previous study (Scheibel and Scheibel, 1958), calcu- 
lations were made in an attempt to determine to what extent the activity of 
the individual reticular neuron might be modulated by adjacent elements. 
Each soma-dendrite sphere, because of the dendrite orientation previously 
mentioned, was assumed to be flattened on its rostrocaudal faces until it 
achieved the dimensions of a cylinder of 100 yu in height (the rostrocaudal di- 
mension) and 200 u in diameter (a conservative estimate of the average length 


of a single reticular dendrite in the brain of a 10-day-old kitten). Average 


cell density in the lower two thirds of the reticular tegmentum of such an 
animal was calculated as 33 cells per 10° cubic microns. In the soma-dendrite 
envelope thus described, and called, for brevity, a. p. ica) (area of potential 
interaction of the soma-dendrite complex), we calculated that 4125 cells would 
be present. It is therefore theoretically possible that a cell aggregate of this 
number might show modifications in their own activity patterns or firing 
thresholds following discrete activity in one cell. 

In cortical tissue, it seems unlikely that a single cell would be activated by 
an afferent barrage due to the extensive arborization of most cortical presyn- 


860 Annals New York Academy of Sciences 


aptic afferents. In the reticular formation where presynaptic termini are 
simpler, when the extracellular microelectrode was sampled from two adjacent 
reticular elements simultaneously, we have often seen one unit driven by an 
input that has no effect, or even an opposite effect, on the other. Direct 
transsynaptic effects probably supercede ephaptic ones, but in those cases 
where one of the pair alone receives contributions from the active fiber, field 
potentials in the surround might become meaningful and the number of ele- 
ments thereby affected might become significant. 


Output Arrangements 


Terminal degeneration studies (Nauta and Kuypers, 1958; Brodal, 1957) 
can only hint at the extensive elaborations of the individual reticular axon 
emitted by a single cell. Golgi silver impregnations in suitable thick sections 
indicate that most reticular cells emit axons that run for long distances rostrad 
or caudad, emitting frequent collaterals as they go. The larger cells, such as 
those of the magnocellular group in the bulb, and many of the more mesially 
placed elements in the pontile and mesencephalic tegmenta are characterized 
by axons that bifurcate into rostral and caudal coursing segments. The total 
lengths achieved by such axons may be quite remarkable. Thus a cell of the 
pontile tegmentum may send its rostral segment through the mesencephalon, 
the entire length of the hypothalamus and into the basal olfactory region and 
septum. The caudal segment may enter the spinal cord in the ventral or 
lateral funiculus, terminating on or about internuncial or dorsal horn cells at 
a number of levels. Along the entire course of the axon, collaterals are given 
off to adjacent portions of the reticular formation, to cranial nerve nuclei, and 
to other sites (Scheibel, 1955; Scheibel and Scheibel, 1958). Because of the 
complexities of this system, it seems preferable to handle each system of pro- 
jections in turn. 

The collateral systems. The number and variety of side branches that the 
reticular axon may give off is surprising. Some are little more than a bouton 
on a stalk of a few y in length, effecting contact with cells (neurons or oligoglia) 
virtually in the path of the main conductor. Others are longer, leaving the 
parent fiber at right angles to wander some distance into the reticular matrix, 
making synaptic contact with more remotely positioned cells. Characteristic 
also of such fibers are the side branches that enter nonreticular nuclei along — 
the way, that is, sensory and motor cranial nerve nuclei and extrapyramidal 
centers. 

In an effort to make some estimate of the degree of interaction of these 
fibers among the surrounding reticular population, a number of measurements 
were made of collateral length, direction, and frequency of branching from 
the main radical in kitten material (Scheibel and Scheibel, 1958). Upon 
averaging the data, we were led to make some simplifying assumptions that 
enabled quasi-quantitative interpretations. We assumed that the typical 
reticular axon emits one collateral in some direction from the parent stalk 
every 100 uw. Each collateral is assumed to be 100 yu in length. We can then 
transcribe a circle of 100 uw radius whose volume we shall call the area of po- 
tential interaction of the axon (4. p. i.a2). Operating from the further assump- 
tion that every reticular cell within this volume may, although not necessarily, 
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be acted upon synaptically by the fiber in question or by its dependences, and 
still working with our previously-derived figure of 33 cells per 10° cubic u 
for average cell density in the lower two thirds of the brain stem, we calcu- 
late that 27,500 cells may lie in the a. . i.az of a single fiber of a conservative 
2-cm. length. On the basis of the average cross section of a 10-day-old kitten 
stem, it can be shown that such a cell population approximates 0.3 per cent of 
the total reticular cell population below the diencephalon. We infer that if 
300 crucially placed reticular axons were simultaneously active, they would 
temporarily commandeer the activities of a reticular population of almost nine 
million cells. 

The possibility that a fiber ensemble thus crucially arranged will be selected 
for activity at one time is improbable. Also there is reason to believe that 
when a spatiotemporal configuration traverses a system, reticular or otherwise, 
a number of parallel channels are in simultaneous use according to the nature 
of the configuration, its intensity, locality, and quality. The number of 
channels for simultaneous transmission of discrete bits of information are thus 
more widely restricted. In the section on input arrangements we noted how 
afferent systems overlap each other as they project upon the reticular core. 
From this point of view, too, competition for circuit space must be active. In 
addition to the restrictions already assumed to be present in this system, data 
from a recent intracellular recording program suggest still another (Scheibel, 
Uchiyama, Uchiyama, and Scheibel, work in preparation). In a series of 
experiments, we noticed that as the intracellular probe was advanced through 
reticular tissue, frequent shifts in resting potential were recorded. These were 
of an amplitude and direction that could be considered to mark the entrance, 
passage through, and exit from successive neural (and glial) components. 
Only one or two such penetrations per hundred resulted in the appearance of 
spontaneous or driven spike activity. Since ordinarily such a preponderance 
of nonneural elements (glia, vascular tissue, and others) would not be expected, 
our only alternative interpretation was that in a neuron pool such as the re- 
ticular formation, only a fraction of the available units are maintained at or 
near those levels of excitability compatible with transmission. The rest must 
exist as a reservoir of temporarily nontransmitting units available for use at 


other times. Operations of this sort would be in poor accord with usually 


accepted theories of “place” specificity in conduction systems and would sug- 
gest, instead, a far more plastic mode of transmission where originating locality 
of the bit was represented in other ways than by strict intrafascicular geog- 
raphy. ‘Thus there is further gain in the over-all flexibility of the system and 
further loss in channel availability at any one time. 

We are left to assume that ability to manipulate‘a number of inputs simul. 
taneously is not a high-priority item for the reticular core. The stacking of 
heterogenous afferents upon the same cell body points the same way. It 
would seem, rather, that the system concerns itself with integrates of afferent 
and efferent data: a sample cross section of the specious present. The nature 
of the reticular output must then reflect the vectored nature of these inputs, 
both as to quantity and quality, and is continuously projected upon higher 


and lower stations in the neuraxis. wie 
Before leaving the general area of collateral patterns, it seems indicated to 
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touch on the problem of “the Golgi type Il component” of the reticular core. 
We have already called attention to the fact that in the reticular system there 
appear to be no true short-axoned cells. However, a significant proportion of 
the reticular components release one or more axon collaterals, close to the 
parent cell body, that break up in elaborate plexus arrangements similar to 
the termination patterns of short-axoned cells elsewhere in the nervous system 
(Scheibel and Scheibel, 1958). The path length of these collaterals is usually 
very short (50 to 200 y) and the terminal elaborations are extensive. Poten- 
tial theory would suggest these as ideal settings for the development of decre- 
menting, nonpropagated fields. If so, then the very fact of activity In a re- 
ticular soma would immediately change field patterns in the immediate area 
of the soma, with consequent changes, in turn, on threshold values of this and 
adjacent cells. The significance of these changes is still unknown, but Fessard 
(1954) offers an excellent theoretical discussion of the properties of such cir- 
cuits. 

The descending systems. Axons from cells of bulbar, pontine, and mesen- 
cephalic levels can be seen in suitable preparations to stream caudally and 
enter the spinal cord. The organization of these systems was first studied by 
Papez (1926) and more recently by Neimer and Magoun (1948), among others. 
The methods of Golgi clearly show how intensively the collaterals and ter- 
minals of these fibers are synaptically involved at sensory relay stations such 
as the gracile and cuneate nuclei, and widely through the dorsal horn of the 
spinal cord. In these areas, the simple reticular presynaptic terminals appear 
to course obliquely through the sensory receptive fields made up of the char- 
acteristic arborizing and overlapping sensory bushes and postsynaptic cell en- 
sembles, effecting contacts here and there, and appearing to fractionate the 
field (Scheibel and Scheibel, 1960). 

Reticular control can also be effected at even more peripheral stations, as 
suggested by the work of Granit (1955), in changing the firing rate of retinal 
cells through mesencephalic tegmental stimulation, and by Herndndez-Peén 
(elsewhere in this monograph), who observed depression of auditory potentials 
in the cochlear nucleus following tegmental stimulation, although data from 
Dell’s laboratory (as reported in these pages) casts some doubt upon the in- 
terpretation of these results. Muscle spindles are also under the control of 
this system, as the work of Eldred and Fujimori (1958) indicates. 

The actual nature of the synapse in the internuncial and ventral horn areas 
is not yet satisfactorily worked out. However, there seems little reason to 
doubt that the pattern would follow that of reticular projections to other 
areas, that is, multiple axodendrite synaptic contacts terminating with or 
without bouton contributions to the synaptic scale of the soma itself. On the 
basis of our study of the reticular synapses in the cortex, we suspect that the 
important transactions occur at the axo-dendrite level. 

The ascending systems. In many respects, the ascending components of 
the system are the most crucial and the most difficult to understand. As 
pointed out by Nauta and Kuypers (1958) the presence of ascending fiber 
systems in the reticular core was known at the turn of the century by Kohn- 
stamm and his associates (1908). However, such data proved of little signifi- 
cance until the work of Forbes and Morison (1939) and of Moruzzi and 
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Magoun (1949) emphasized the importance of this projection in parallel with 
the classical sensory systems. Recent investigations clearly indicate that an 
ascending axonal stream from reticular cells of bulb, pons, and mesencephalon 
can be traced at least as far rostrad as the anterior mesencephalic tegmentum. 
Here splitting occurs just behind the centromedian-parafascicular complex, 
one leaf projecting through the medial thalamus in the intralaminar zone 
while another descends ventrolaterally and continues rostrad in the zona in- 
certa of the hypothalamus (M. Scheibel, 1955; Brodal, 1957; Nauta and 
Kuypers, 1958; Scheibel and Scheibel, 1958). The ultimate fate of these 
fibers is still not clear, and the Golgi methods have thus far given us only 
partial answers. The ventral component collateralizes widely in the incertal 
region and many fibers pass as far rostrad as the septum, basal olfactory area, 
and striatum. Because of the technical problems involved, we have not yet 
béen able to establish whether any of these axons continue without synapse 
into the archicortex and neocortex. A number of them seem almost certain 
to collateralize or terminate in the ventral portion of the nucleus reticularis 
thalami and possibly the ventral anterior nucleus of the thalamus, while others 
are lost among the masses of preoptic and supraoptic hypothalamic cells. 

Fibers of the dorsal component run anteriorly through the intralaminar 
system of nuclei, collateralizing widely into the adjacent sensory relay and 
associative nuclei of both sides. Many of them terminate in the anterior 
thalamic nuclei, or nucleus reticularis thalami, but some seem to perforate 
these and project at least into the striatum. Once again, the total extent of 
these projections is not known. 

A second nonspecific system is clearly shown by fortunate Golgi sections to 
originate in the centromedian-parafascicular complex, run forward in parallel 
with the former system on its medial side, and collateralize, even more abun- 
dantly, to almost every thalamic site. This upper reticular component ap- 
pears to be the one found by Sharpless and Jasper (1957) to mediate phasic 
components of consciousness in contradistinction to the tonic control exercized 
by the lower component. Once again, the rostral limits of this upper system 
remain to be worked out, but we are certain that many of these fibers collat- 
eralize and/or terminate in the ventral-anterior nucleus and nucleus reticularis 
thalami. It does not seem unlikely that axons of this system may continue 
rostrally as reticulocortical components. 

Due to the contributions of both of these systems, and to the rich collateral 
addition from descending corticoreticular fibers, a heavy neuropil plate is 
generated in the nucleus reticularis thalami surrounding dorsal, ventral, and 
rostral aspects of the thalamus and in the adjacent anterior thalamic nuclear 
complex. Through this plate and the external medullary lamina applied to 
the inner surface of the internal capsule pass virtually all the thalamocortical 
fibers and most of the corticothalamic reflux. We have already suggested the 
analogy between this and the screen grid in a vacuum tube (Scheibel and 
Scheibel, 1960), thus exerting possible modulatory effects on all commerce be- 


tween diencephalon and cortex. ‘ , 
We outlined the probable nature of the reticulocortical relay in previous 


publications (Scheibel and Scheibel, 1958, 1960). It seems likely that the 


nonspecific fibers terminate along the apical dendrites of cortical pyramids in 


864 Annals New York Academy of Sciences 


long-climbing axodendrite appositions. In view of the work of Claire and 
Bishop (1955), supporting a nondigital nature for cortical dendrite electrical 
phenomena, they may serve as bias adjustors, continuously resetting the ex- 
citability thresholds of cortical somata and initial segments. Since the non- 
specific relay serves as a principal contributor of presynaptic endings to these 
shafts, it would seem that the reticular core exerts an essentially analogical 
control over the digital (all-or-none) output sequences of the cortical cells. 

In addition to controlling cell excitability levels, it was suggested also that 
these fibers might be involved in organizing somatotopic representation at the 
cortical surface. It was found that repeated subgriseal bifurcations of the 
nonspecific fibers caused each to send a number of fibers toward the surface, 
so arranged in relation to each other as to form a cylinder whose height was 
the thickness of cortex and whose diameter was of the order of 1 mm. (Schei- 
bel and Scheibel, 1958). This was also the figure given by Mountcastle (1957), 
who found that individual cortical neurons within such a radius usually re- 
sponded to driving from only one point in the periphery. We suggested that 
this type of topical specificity might be due to reticular modulation of the 
cortical cell threshold to one specific area rather than another. Certainly the 
overlapping of sensory afferents was sufficient to cause wider driving effects 
than this. 

Our knowledge of the structural organization of the reticular core, incom- 

plete as it is, leaves us with an image with which it is possible at least to work. 
We conceive of a system whose input-output arrangements allow it to lie 
athwart all incoming and outgoing systems of the brain. Through every mo- 
ment in time it receives continuous samples of information being relayed in 
these systems, and this information is fed heterogenously into the cell-fiber 
matrix of the core. While qualitative specificity must be lost in the integra- 
tive processes that occur, it is suggested that quantity and change of influx 
are important parameters in determining the output of the system. While 
the system seems arranged to vector its inputs so that its output is always a 
function of them severally, it is also known to be extremely fickle, adapting 
to a stimulus as it becomes more familiar. Thus, competition for input su- 
premacy in the core must rest on more than the vagaries of local geography 
and the laws of statistics. The fresh and the unfamiliar are more than likely 
to usurp reticular activity from the stale and predictable. The less familiar 
(hence less predictable) the sensory or motor sequence, the more its outcome 
may matter to the organism. In fact, the reticular core seems primarily con- 
cerned with the problem of mattering, or the investment of a neural event 
with significance, without which, no matter how available the circuitry, the 
event is apparently neither experienced nor remembered. 
The unique properties of this system, some already demonstrated, others 
inferred, appear to deserve special emphasis in a monograph on human de- 
cisions. Although.the very decision to include this system may in itself finally 
appear to have been in error, it would serve to underline a basic dynamic of 
human decision making: continuous formulation and shaping as the problem 
is engaged and progressive redefinition and focussing down as further informa- 
tion modulates that already being processed. 
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THE EFFERENT CONTROL OF AFFERENT SIGNALS ENTERING 
THE CENTRAL NERVOUS SYSTEM 


R. Hern4ndez-Peén, H. Brust-Carmona, J. Pefialoza-Rojas, G. Bach-y-Rita 
Unidad de Investigaciones Cerebrales, Centro Médico del Distrito Federal, Mexico, D.F. 


The centrifugal control of afferent transmission in central sensory stations, 
long suspected by the perceptive neuroanatomists Cajal,” Brouwer,’ and Wal- 
lenberg,” is now firmly established as the result of numerous experiments car- 
ried out during the last six years. 

Early as well as recent anatomical studies,?:?5°.%.%4.40.4. have shown in 
the central nervous system the presence of efferent or centrifugal fibers pro- 
jecting down to sensory synapses as low as those of the spinal cord, of the 
spinal fifth sensory nucleus, of the dorsal column nuclei, of the cochlear nu- 
cleus, of the retina, and of the olfactory bulb. It has also been demonstrated 
in acute experiments that electrical stimulation of these fibers elicits inhibi- 
tory! 219 or facilitatory"! *!10 interactions with the ascending afferent 
impulses in all the above mentioned sensory centers (see the reviews on this 
subject by Hagbarth!* and by Herndndez-Pedn'*). 

Furthermore, some descending influences that control synaptic afferent 
transmission act tonically during wakefulness. It has been shown that syn- 
chronized afferent volleys recorded from the lowest levels of the somatic, audi- 
tory, and olfactory central pathways are enhanced by anesthetics and by brain 
stem lesions, which impair consciousness.!*:?9!°.27_ These experimental obser- 
vations are consistent with older observations in which larger cortical sensory- 
evoked potentials were recorded during deep rather than during light anes- 
thesia.8 Therefore the conclusion is warranted that the reticular activity 
related to wakefulness normally regulates the passage of a great number of 
sensory impulses to the central nervous system. Undoubtedly this new neuro- 
physiological concept can account, at least in part, for the well-known psycho- 
logical phenomenon concerning the constantly limited span of perception. 

Since perception is invariably associated with the process of attention, it 
occurred to one of us (R.H.-P.) that sensory impulses generated by stimuli 
outside of the span of attention would be prevented from entering the brain 
by descending inhibitory influences such as those mentioned above. Corre- 
lated behavioral and electrophysiological studies carried out in unanesthetized 
cats with electrodes permanently implanted in the lowest subcortical sensory 
relays have substantiated this hypothesis. By recording synchronized afferent 
volleys evoked by brief sensory stimuli, Herndndez-Pe6n and his co-workers" 
have demonstrated that the corresponding evoked potentials are reduced or 
abolished when the animal attends to stimuli more significant than the test 
stimulus. This phenomenon has been observed in the dorsal cochlear nu- 
cleus,* in the axons of the retinal ganglion cells along the optic tract,22°3.7 in 
the spinal fifth sensory nucleus,’ in the lateral columns of the spinal cord,* and © 
in the olfactory bulb.!® A similar reduction of sensory-evoked potentials at 
those lower stations of the mentioned sensory pathways has also been observed 
by Hernandez-Peén and his co-workers!® consecutive to monotonous repetition 
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of the same sensory stimulus. This phenomenon represents an electrical sign 
of the learning process, termed by Dodge “habituation,” and we shall refer 
to it as afferent neuronal habituation. In all the former experiments the re- 
cordings were made exclusively from one sensory pathway at a time; therefore 
only the behavioral response of the animal to other stimuli was observed. In 
order to obtain a more complete view of how the waking brain controls the 
entrance of the various signals coming from sensory receptors, we decided to 
record simultaneously the spontaneous and the evoked electrical activity from 
the dorsal cochlear nucleus, from the optic tract, from the olfactory bulb, and 
from the spinal fifth sensory nucleus, as illustrated in FIGURE 1. 


on PX 


: ; 5 oe t ae 
Ficure 1. Diagram showing the recording sites from lower-order sensory neurons in 
olfactory (1), visual (II), trigeminal (V), and auditory (VIII) pathways. 


Material and Methods 


The electrodes were stereotaxically implanted, as described elsewhere,?®.”” 
and the recordings were made with an ink-writing electroencephalograph. 
The test stimuli consisted of clicks, flashes of light, single electrical shocks 
delivered to the skin of the face, and puffs of air conveying fish odor. The 
different stimuli were delivered simultaneously or alternatively in patterns to 
be described later. Unless otherwise stated, the intensity of the test stimuli 
was kept constant throughout the entire experiment. 


Results 


Effects of alerting stimuli upon the electrical activity of lower order sensery 
neurons. ‘The outstanding feature of the changes observed in the spontaneous 
and the evoked electrical activity in all lower sensory stations during eetpe 
induced by different kinds of sensory stimulation was the great variability o 
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that activity as compared to the very stable evoked responses recorded from 
the relaxed animal. Ficure 2 illustrates this effect during simultaneous rec- 
ords taken from the olfactory bulb, from the trigeminal nucleus, and from the 
cochlear nucleus. When the cat was awake but relaxed, the electrical activity 
of the olfactory bulb was minimal and uniform, and the tactile and auditory 
evoked potentials were approximately constant in size. When a mouse was 
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presented to the cat, significant changes of the electrical activity in those sen- 
Sory structures occurred during the behavioral alertness of the animal. At 
first the auditory potentials diminished in amplitude, the tactile trigeminal 
potentials were facilitated, and bursts of rhythmic activity as described by 
Lavin et al.** and by Herndndez-Peén et al.26 began to appear in the olfactory 
bulb. A few seconds later, when the cat seemed to be attentively looking at 
the mouse, the tactile trigeminal potentials were practically abolished and the 
auditory potentials were further reduced, while the arousal discharges from the 
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olfactory bulb reached their maximum amplitude. Two minutes after re- 
moving the mouse from the cat, the electrical activity of the olfactory bulb, 
as well as the potentials recorded from the trigeminal nucleus and from the 
cochlear nucleus, regained their original amplitude. The reduction of the 
sensory-evoked potentials was maintained for a few seconds only as the cat 
appeared to be attentively looking at the mouse. As shown in FIGURE Sh 
when the cat was alert without being specifically attentive to a given object, 
the size of the evoked potentials varied intermittently from one moment to 
the next. These changes occurred simultaneously or alternatively in different 
sensory pathways. 

Arousal reaction consecutive to cessation of repetitive stimulation and its mani- 
festations in sensory pathways. An observation of everyday life is that arousal 
may be produced upon cessation of sleep-inducing monotonous stimulation. 
In’ order to study the electrophysiological correlates of poststimulatory arousal 
in sensory pathways, we decided to record the changes of the electrical activity 
in the various lower sensory synapses upon cessation of a series of monotonous 
clicks. The presentation of the first clicks induced the classical orienting 
response, and by repeating these clicks the animal became relaxed and drowsy. 
If at this moment the acoustic stimuli were suddenly stopped, the animal be- 
came aroused and high-voltage bursts of rhythmic activity appeared in the 
olfactory bulb. Poststimulatory arousal was rapidly habituated, as shown in 
FIGURE 4, just as the arousal reaction induced by presentation of a novel 
stimulus can be habituated. 

The arousal response elicited by cessation of the monotonous series of clicks 
was no longer obtained when the procedure was repeated. It is evident, there- 
fore, that habituation of poststimulatory arousal can be produced by one 
single cessation of intermittent stimuli. 

Sometimes we also observed a transient reduction of the evoked potentials 
recorded from the trigeminal nucleus or from the optic tract during the first 
two or three seconds of poststimulatory arousal (FIGURE 5). : 

Afferent neuronal habituation in lower levels of the tactile, auditory, and visual 
pathways. The phenomena of habituation and dishabituation by diaphoro- 
stimulation already described in various sensory pathways’ were readily con- 
firmed in our experiments. In addition to recording the potentials elicited by 
the habituating stimulus, we recorded the dishabituating impulses producing 
disinhibition in another sensory pathway, that is, heterosensory hyperrespon- 
siveness. F1cuRE 6 shows that tactile habituation recorded from the spinal 
fifth sensory nucleus was easily disrupted by the sudden entrance of photic 
signals simultaneously recorded from the optic tract. Conversely, photic 
habituation recorded from the optic tract was released by sudden application 
of tactile stimuli (FIGURE 7). 

Simultaneous recordings of evoked potentials from several afferent pathways 
showed different rates of habituation for the various sensory modalities. Tac- 
tile habituation always occurred faster than that elicited by acoustic or photic 
stimuli, as illustrated in FrcuRE 8. After the establishment of photic eee 
tion by monotonous repetition of flashes of light, suddenly applied tactile and 
acoustic stimuli resulted in the reappearance of the photically evoked poten- 


Annals New York Academy of Sciences 


870 


‘sSau}io[e SULINP pUk SSoU[NJoyeM poxe[el SUI 
“inp 38) ® wooly papiovar ‘(*N"D) Snaonu Avayooo ay} wWIoIy pue ("g’O) q[Nq A10}9eI[O ay} Woz APATIOV [eOUWDIT “¢ TANT 


‘90S | 
Se WILS J1LSNOdV =F 


v Tv v ’ v v “ef 5 | ae | 2a 
‘ato phalplvalsteello Awvtntea til eo tienen AA nell aserian/batnet eat ‘N'O 


: | 


"A O5 DPhil ot B= Me) 
Lt1ua3aid1y 


‘AM OND vw Loisdea od ride toyp SoReal ane ‘NO 


afos 3 


~ 


aqaxviau. 


871 


Hernandez-Peon et al.: Control of Afferent Signals 


“SUOISBIIO 2ATJNIISUOD OA} UO ‘S¥YDI[D JO SdLIas SnouUo}OUOU B Jo UOT 
-essao Aq poonput ‘(*N"d) sheponu eepypoo oy} Jo pure (“g'O) q[Nq A10;9RJ[O dy} Jo APATIO" [eoMzOaIJ9 OY} UI SasueyD “fF TANOIT 


‘00s | 
—— WILS DILSNOOV 


2 vse v v 


| ae 
[Maal DAN Papnteveh mcr haba" atnl weasel soy santas ey NO 


ator J 


poet aftr NCO Vth hea ANAM rh ay macy 
afos J 


I 
‘a0 


‘aon ) concn spon et CA fe ‘NO 
atos | te fieraatrtnalgntntnrnen ner stuiyri, AAYrmetninpnetnie tnt 0 


Annals New York Academy of Sciences 


872 


*T[NUUT}S IT}snNode 
ay} Sundnizajzur 1038 ssaujiaye Jo spuosas xis Surmp syerjuajod otoyd ay} Jo uoT}ONpar B aJ0N “SHOT Jo Satias snouojou 
“OU B JO WOTVessad SuIMOTOF ‘(\L°O) 30817 9FNdo ay} Wo. pepsodal syerjuaj}0d peyoaa-orjoyd ay} ulsasueyd *¢ ANNO 

29S =6S'0 WILS SILSNOOW =F 


SS 
WILS SJILOHd © 


ado J ONION rr Nt otal en DIA fA PA Ne pe pope N'D 


2 “ ; Q3xv7134y 
‘ado J Dea rure\ otal pennant oe hel ns rn (ean) Pg tmp ER ‘LO 


‘ado y RAL It Pt nap teenth, reine ND) 
é e e L¥3aT1V 


ato td Me aad ne antanteaea Via nial sateen et een eT I TERT Tee) 


v v j ¥ 
ato l seneaiedl pemteteentatend eetetemmenen aed ean, 9 
e ® e aQ3xvi13y 


‘afol mre panne nara termnl pa reewtrnnanrennereni ln ricren “LO 


873 


Hernandez-Peén et al.: Control of Afferent Signals 


CLO) eI Ido ay} wWoIy papsooal ‘sferyusjod payoss yey) YSI Jo saysey Jo uoNe}Ueseid uappns oy} Aq psonpur 


uorenyiqeysip pue “(N’L) snapnu Arosuas yyy [eurds oy} wos, popsooaI sfetjusjod a[1j9e} Jo uoNeNyIGeE, “9 AUNNOIA 


‘A® 08 


‘A® O8 


‘Am Od 


‘At OS 


“Af 08 


“AP 08 


WLS 3NLOVL Oo 
WILS DILOHd © 


‘98s GO 


Qa 
a 
: Sc hia -  @3.LVNLIGVHSI 


| 
Pte a lanrtntiirtale as tant manele pn ure “LO 


2O 320 U 
I 


as) «Oo vou ‘ho ke 
1 synthe nt ae a Myre pyre tnen thant Mn nian INL 
; --  @3.LVALIGVH 
DeAnna sini areas ° lO 


Bo meal regen hey hal jet dante | tam rneniA 
. TOH¥LNOD 


| Prraenata Nenmmnyt anita ayn Peayslaynnun\y AM alirnennr “LO 


Annals New York Academy of Sciences 


874 


atoz | 


‘aftoz J 


atoz j 


atoz | 


“CNL) shaponu jeururas113 oy} wory paps0dai speruajod payoaa YoIyA ‘NWS 9[1}9e} Jo UoTyeyUasaId 
Aq worenyiqeysip pue ‘(\1'Q) 39e1} odo ay} Wosy papsozer sjetjusjod ooyd Jo uoNnenyqeyy “yf AUNT 


‘29s GO WILS 31L0VL O 
WILS OIILOHd @ 


Q3LVNLISVHSIG 


Q3LVALISVH 


TOYLNOD 


875 


Hernandez-Peon et al.: Control of Afferent Signals 


‘pesueyoun pourewial spetjuajod Aroyipne pue s1j}0y4d 343 
aya ‘psonpai sam spetUuajod jeurmest1} ay} NTs AJo}IpNe pue a[t}Ie} QZ Joie yey} smoYs ) “SHIP sev Pe se 
‘TeUMTUR Bq} JO VdLF IY} 0} T[NwUT}s o[1}9e} Burars Aq payor sjerusjod payoas-o1j0yd oy} Jo WoTeI0jSa1 BY} S}USSeIdel g 
“py ULUMOYS SI YS] Jo soysep OOP 10772 speruajod oNoyd 94} jo uoTeNzIGeyY “("N°D) Sneponu Teayysoo out woiy pue 
‘CN’L) Snepnu Arosuas yyy peutds oy} wor ‘(L°Q) Joe} Odo oy} Woy SZUIPIOIII SNOIUe}[NUIS “g AXA, 


WILS O1LSNOOY ov 
peo geo. WLS 37IL9vL 9 
be tebe 


WILS JILOHd @ 


Zs ——_ 
‘atoz | pe pe entireties [Nm J ‘N'O 
‘atoz | PP Pe NO ER MIT ME SN ces NL 


é & 


o o o 0 = 
atoz J manera parocmarenrcornamca) \werer aorta ethan tan aronstcn tmtined rast Wd 
a : : ‘ : st 


c) ® | 


. 
‘AW O08 | parton and vata, fon neertinn Ay Al ad an ee tren din pon NSA etn n/a Mn pre" “SLO 


ae sae a 


876 Annals New York Academy of Sciences 


tials in the optic tract. Shortly thereafter, during the simultaneous three- 
modality stimulation, the tactile potentials were significantly reduced in con- 
trast with the photic and auditory potentials, which remained unchanged. 
Another typical effect is shown by the experiment illustrated in FIGURE 9, in 
which acoustic and tactile potentials were recorded simultaneously from the 
bulbar sensory relays. After a few dozen stimuli, the trigeminal potentials 
almost disappeared, whereas the auditory potentials remained unchanged. At 
this moment the application of a visual stimulus that alerted the cat brought 
about the reappearance of the trigeminal potentials (dishabituation), as well 
as a diminution of the auditory responses. In the experiment illustrated in 
FIGURE 2, the application of a given stimulus simultaneously elicited blocking 
of one sensory pathway and disinhibition of the other afferent system. The 
opposite effects are obviously related to the previous state of the sensory syn- 
apses involved. 


Comments 


Afferent transmission during alertness. The above mentioned experimental 
results provide direct evidence for the great versatility of the centrifugal sys- 
tems that regulate sensory transmission during wakefulness. Facilitation of 
the electrical activity in one sensory pathway is always accompanied by block- 
ing of sensory transmission in other afferent pathways, and this phenomenon 
occurs alternatively from one moment to the next in the various sensory sys- 
tems. It seems that since the amount of information the waking brain can 
manage at any given moment is limited, a selection is accomplished by facili- 
tating sensory transmission for the meaningful afferent signals and by simul- 
taneous blocking of the remaining incoming signals to the central nervous 
system. 

In discussing the neural mechanisms of alertness a distinction should be 
made between unspecific alertness, which often accompanies an inquisitive 
attitude on behalf of the animal, and specific alertness, characterized by a state 
of attention focused upon a given stimulus. In the first situation the de- 
scending or centrifugal influences that regulate sensory transmission act inter- 
mittently and in random fashion. As soon as attention is directed and focused 
upon something, the descending systems for sensory control maintain an or- 
ganized activity. 

In contrast with the great deal of variations observed during alertness in the 
electrical activity of sensory pathways, there is a great stability in the same 
sensory structures during relaxed wakefulness or sleep. These findings suggest 
that the centrifugal system involved in sensory regulation is related to the 
“arousal system” whose essential part is located in the mesodiencephalic region 
of the brain stem. 

It may be argued that the apparently diffuse projections of the reticular 
system cannot account for the organized selection of sensory impulses during 
attention. However both anatomical and physiological studies have shown 
a complex organization in the brain stem reticular formation, which is not as 
diffusely arranged as originally thought. Sensory convergence in the different 
elements of that region is only partial! 25 and the specific information received 
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by the reticular neurons is preserved, as shown by their patterned activity 
evoked by the same sensory stimulus.! 

As postulated in FIGURE 10, it seems likely that the feedback circuits linking 
cortical and reticular neurons to specific sensory elements are topographically 
organized so as to influence selectively the incoming sensory information. In 
addition to the corticoreticulo-sensory system, each specific afferent pathway 
is accompanied by a chain of neurons carrying impulses in the opposite direc- 


SPECIFIC AFFERENT PATHWAY 


PARASPECIFIC DESCENDING PATHWAY 


CORTICORETICULO-SENSORY PATHWAY 


‘ Ficure 10. This diagram illustrates the descending pathw h i 
mission at lower sensory synapses. es Be that en ee 


tion, which may be termed paraspecific descending pathways. The effects of 
electrical stimulation of the latter system are now being studied by C. Ajmone- 
eran (personal communication, 1960) and in several other laboratories.®?:3*) 

Although there is yet no clue about the functional role of the extrareticu- 
lar descending pathways, we venture to propose as a working hypothesis that 
the afferent control achieved by them might serve for fine sensory discrimina- 
tion resulting from highly elaborated learning processes. 

Poststimulatory arousal. Drowsiness or sleep induced by monotonous repe- 
tition of the same stimulus in a quiet environment is a common and intriguing 
observation whose mechanism remains obscure. Controversial interpretations 
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have been centered either on the reduction of stimuli, or on an active inhibi- 
tory process as the responsible factors for somnolence. 

According to the first hypothesis translated into modern neurophysiological 
concepts, sleep would be the passive result of diminished activation of the 
brain stem arousal system, which is constantly impinged upon by a great 
variety of sensory impulses. The second hypothesis implies that rhythmic 
sensory stimulation progressively recruits inhibitory neurons acting upon the 
arousal system. 

Cessation of monotonous stimulation provided an experimental procedure 
for testing the two above mentioned hypotheses. If sleep were passively de- 
termined, discontinuation of stimuli should enhance the depth of sleep. On 
the other hand, sudden interruption of inhibition usually elicits rebound ex- 
citatory effects that in this case would be manifested as arousal. Our experi- 
mental results are consistent with the latter hypothesis. 

The recent discovery of an EEG synchronizing region caudally located in 
the brain stem also supports the view that sleep is the result of active inhibi- 
tion. In experiments designed to test whether the bulbar synchronizing struc- 
tures are responsible for the EEG synchronization induced by the sheer rep- 
etition of photic stimuli, Mancia ef al.*® have repeated this experiment in 
permanently awake cats with the brain stem transected at the midpontine 
level. These authors found that EEG synchronization can still be induced in 
the midpontine cats by applying a larger number of photic stimuli than those 
required in intact animals. Such experimental results indicate that in the 
intact brain the postulated bulbopontine inhibitory ascending reticular forma- 
tion participates partly in the monotonously induced sleep. It seems likely 
that regular intermittent sensory stimulation also activates mesencephalic and 
supramesencephalic neurons, which are inhibitory for the mesodiencephalic 
elements of the arousal system. 

There are suggestive data indicating that such sleep-inducing descending 
inhibitory influences might be mainly located within the limbic system. Both 
bulbopontine ascending and limbic descending inhibitory neurons, which would 
be the anatomical substratum of the “‘sleep system,”’ were discussed with the 
problem of habituation of arousal by Herndndez-Pedn in the Moscow Collo- 
quium.!® : ; 

Plastic nature of habituation and dishabituation. Habituation and dishabit- 
uation occurring at the first sensory synapses have been discussed in detail 
from the neurophysiological standpoint in several papers.'°47 The cumula- 
tive and long-lasting effects of repetitive sensory stimulation are manifesta- 
tions of plastic inhibition, which must be distinguished from the transient 
sensory depression related to attention. Although dishabituation by diaphoro- 
stimulation can be interpreted as “inhibition of inhibition,” it is also evident 
that the persistent effects of the dishabituating inhibition are not comparable 
to the transient depression prcduced by the same stimulus upon nonhabituated 
sensory-evoked potentials. ~ 

From our experiments it became clear that the same stimulus can elicit 
different effects in any given sensory pathway according to the previous ex- 
perience of the animal. Therefore, if that experience Is not taken into con- 
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sideration, dishabituation can easily be misinterpreted as simple sensory facili- 
tation. 

It can be concluded that the central nervous system is endowed with com- 
plex mechanisms that permit the selection of a limited amount of sensory 
signals long before they reach the higher integrating structures, and that this 
selection is made according to the significance of the stimuli and to the past 
experience of the organism. 


Summary 


In an attempt to study the simultaneous control of the various sensory im- 
pulses entering the waking brain, a series of experiments were performed on 
cats with bipolar electrodes permanently implanted in the olfactory bulb, in 
the optic tract, in the dorsal cochlear nucleus, and in the spinal fifth sensory 
nucleus. Simultaneous electrical recordings of the background and sensory- 
evoked activity from these structures provided the following results: 

(1) In contrast with the rather stable magnitude of the various sensory- 
evoked potentials recorded during relaxed wakefulness, these potentials under- 
went a great deal of variation during alertness. When the cat became alert, 
the olfactory bulb activity and the trigeminal-evoked potentials increased, 
whereas the acoustic potentials diminished. Later, during focusing of atten- 
tion, both the acoustic and the trigeminal potentials were reduced. 

(2) Regular repetition of intermittent stimuli led to a progressive diminu- 
tion of the evoked potentials in all sensory pathways (afferent neuronal habit- 
uation). 

(3) The presentation of a stimulus that induced behavioral alertness brought 
about an enhancement of the already diminished evoked potentials (dishabitu- 
ation) and a reduction of the larger potentials. 

(4) Cessation of a monotonous series of clicks, which had induced drowsi- 
ness, brought about behavioral and electrical manifestations of arousal in the 
sensory pathways. 

From the above-mentioned results it can be concluded that any given stimu- 
lus can elicit different changes in the lowest synapses of sensory pathways 


according to the significance of the stimulus and to the past experience of the 
animal. 
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PATTERNS OF ACTIVITY OF SIMULTANEOUSLY RECORDED 
NEURONS IN MIDBRAIN RETICULAR FORMATION 


Vahe E. Amassian, Josiah Macy, Jr., Hardress J. Waller 
Albert Einstein College of Medicine, Yeshiva U: niversity, New Vork, N. Y. 


Introduction 


The brain stem reticular formation is a particularly appropriate portion of 
the nervous system to discuss in this publication. Since the work of Magoun 
(1952), the reticular formation has come to be regarded as a major coordinat- 
ing system between the afferent inflow and the efferent outflow. A general 
discussion of the neurophysiological literature related to the reticular forma- 
tion is made unnecessary by the thorough reviews of Segundo (1956), Rossi 
and Zanchetti (1957), and O’Leary and Coben (1958). Our interest in the 
midbrain reticular formation stemmed from the observation that single reticu- 
lar neurons often had wide peripheral receptive fields that extended over 
several limbs (Amassian and DeVito, 1954). However, altering the site of 
peripheral stimulation usually changed the pattern of evoked firing in a given 
reticular neuron. These experiments were done on cats under chloralose 
anesthesia. The analysis was subsequently extended by experiments on lo- 
cally anesthetized cats (Amassian and Waller, 1958). It was proposed that 
different portions of the body were represented in the reticular formation by 
different temporal patterns of discharge of wide receptive field neurons. At- 
tempts to separate the ‘‘position’”’ code from the “‘intensity” code led to the 
conclusion that the distribution of activity within a group of reticular neurons 
following stimulation of one limb could not be matched by adjusting the in- 
tensity of a stimulus applied to another limb. Our attention was directed 
toward the possible importance of the intervals between the first discharges 
of different neurons as a mode of representation of the periphery. Further 
investigations required the simultaneous recording of the activities of several 
reticular neurons that were situated in a limited volume. A technique was 
recently developed for simultaneously recording from three or more cortical 
neurons by three independently manipulated electrodes (Amassian ef al., 1959). 
We inserted only two electrodes at a time into the midbrain reticular forma- 
tion but aimed at recording two separable units with one electrode and at least 
one unit with the second electrode. Our analysis of the preliminary data was 
greatly facilitated by use of a simple, highly flexible computer whose over-all 
operation is described below. 


Methods 


Many of the experimental procedures were previously reported and require 
no further mention (Amassian and Waller, 1958). Briefly, cats were surgically 
prepared under ether anesthesia. Ether anesthesia was subsequently discon- 
tinued; the animal was paralyzed with Flaxedil and artificially ventilated with 
100 per cent oxygen. The cut skin margins and deeper tissues were infiltrated 
with 1 per cent procaine hydrochloride. The micropipettes were filled with 3 
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M NaCl instead of 3 M KCl as in our previous experiments. One electrode 
was inserted in the vertical axis of the sterotaxic apparatus. The other elec- 
trode was slightly inclined so that the two electrode tips approached one an- 
other in the midbrain tegmentum. With practice, a unit recorded by one 
electrode can readily be held during a search for a unit with the other elec- 
trode. However, simultaneous manipulation of both electrodes is often re- 
quired. At the termination of the experiment, a coarse micropipette was 
inserted into the lateral portion of the midbrain down to a convenient depth. 
The brain was subsequently fixed in 10 per cent Formalin in saline and frozen- 
sectioned at 40 p. Sections were stained with cresyl violet. The bottom of 
the track produced by the crude “marker” pipette was used as a reference 
point in localizing the position of the tip of the microelectrode (cf. Amassian 
and DeVito, 1954). The simultaneously recorded units discussed below were 
recorded in the midbrain reticular formation and were situated about 2 to 4 
mm. from the midline. The units were dispersed vertically between the level 
of the aqueduct and the upper border of the red nucleus. 


Analysis 


Neuronal activity was recorded on multichannel magnetic tape, using the 
microelectrode techniques described above and conventional amplifying equip- 
ment. On playback of this tape, the unit discharges were amplified, filtered, 
and center-clipped to remove base-line noise and fluctuations. The spikes 
were then employed to trigger standard pulses of 1-msec. duration and con- 
stant amplitude. Selection and suppression or canceling devices made it 
possible to produce a separate train of standard pulses for each unit recorded, 
even when two units were picked up by the same microelectrode. 

These standard pulses were then passed to a logic unit, and the pulses that 
the logic unit allowed to pass were counted by one of a battery of electronic 
counters. The logic unit generated various functions related to the spikes or 
to the stimuli and used these functions to select the pulses to be passed to the 
counter. 

Interspike interval distributions were constructed by producing a gating 
pulse (the “extension gate’’) initiated by a spike. This positive pulse ended 
after a preset time unless another spike arrived before the expiration of the 
pulse. In this case, the timing was started again from this latter spike, thus 
extending the pulse. This positive pulse was used as a gate, allowing only 
those spikes to be counted whose preceding interspike interval exceeded the 
preset time. A series of such runs with different gate settings produced the 
interspike interval distribution by subtraction. 

A gate that allowed spikes to be counted only during a 10-msec. interval, 
occurring at an adjustable time after the stimulus (‘‘traveling gate”), was used 
to produce the spike frequency distributions. 

“Coincidences” were determined by a gate that allowed only those spikes 
on one channel to be counted that occurred within a fixed number of milli- 
seconds after a spike on another channel. In this paper a coincidence always 
has a number attached: coincidence within T milliseconds. F1curEs 4 and 5 
show plots of coincidences observed as a function of the value of T. 

The expected random coincidence value was calculated by dividing the total 
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time available for coincidence (tc) during the run by the total duration of the 
run, and multiplying by the total number of discharges of the unit. The total 
time the gate was open (tc) was directly determined by counting 1000/sec. clock 
pulses that were led through an identical gate. Superimposition of spikes from 
two units on the same recording channel was missed by the computer, but was 
detected on an oscilloscope, was insignificant as a source of error, and was 
therefore not included in ricurE 4. 


Spontaneous Activity 


Many midbrain reticular neurons are tonically active, but the intervals be- 
tween discharges are irregular (Amassian and DeVito, 1954; Amassian and 
Waller, 1958). Using the extension gate, the distribution of intervals was 
determined during spontaneous activity. One kind of distribution is shown 
in’ FIGURE 16. Note that most intervals are brief; 94 per cent are less than 
200 msec. Although intervals longer than 60 msec. are progressively less 
common, their incidence does not show an exponential decline, and the in- 
cidence of intervals briefer than 40 msec. shows a sharp decline. If events 
are assumed to be randomly and independently distributed in time, so that 
the probability of one event occurring in an interval of time dt approaches 
d dt for some constant X, as dt approaches 0, then the distribution of intervals 
between events can be shown to be an exponential function, and the events 
themselves will have a Poisson distribution (Feller, 1950). This assumption 
of independence of course makes no allowance for refractory periods; in fact 
the exponential distribution of interspike intervals shows that the extremely 
brief intervals are most probable. The assumptions given above can be modi- 
fied to include an exponential recovery of responsiveness for the recorded unit, 
after firing, or for the excitation pathway (cf. Martin and Branch, 1958). The 
results of such a calculation are given in FIGURE 2a; this-shows that even the 
assumption of a rather prolonged exponential recovery does not yield an inter- 
spike interval distribution that matches the observed distribution. However, 
if a sigmoid recovery curve is assumed, as in FIGURE 26 the resulting interval 
distribution is much closer to the observed. The majority of midbrain reticu- 
lar units are not monosynaptically related to the afferent inflow. A sequence 
of relays, each assumed to have a short absolutely unresponsive period followed 
by an exponential recovery, can produce an over-all recovery curve for the 
sequence that resembles the sigmoid curve used in FIGURE 2b. Attempts to 
fit the observed distributions accurately are of limited value at present, since 
so little is known of the recovery of excitability of reticular synapses. The 
data presented here, however, show that even the irregular spontaneous dis- 
charges cannot be assumed to be independent random events. 

A unit recorded simultaneously with the unit of ricuRE 1 yielded a quite 
different distribution of interspike intervals (FIGURE 3b). The neuron dis- 
charged at a very low over-all frequency of 0.6/sec., more than 60 per cent of 
the intervals exceeded 200 msec., (cf. 6 per cent in FIGURE 16) and 30 per 
cent of the intervals exceeded 1 sec. in duration. Nevertheless about 25 per 
cent of the intervals were less than 80 msec. The distribution of spikes is 
‘totally unlike a Poisson distribution, and the interspike interval distribution 
resembles neither an exponential curve nor the modified curves of FIGURE 2. 

The distribution of intervals of both the units illustrated in rrcures 1 and 
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3, was altered by stimulation of the ipsilateral forepaw. The shift toward a 
higher percentage of brief intervals is very obvious 1n FIGURE 3a. By con- 
trast, in FIGURE 1 the percentage of intervals briefer than 40 msec. Is markedly 
reduced by stimulation (these shifts presumably reflect facilitation and depres- 
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FicureE 1. Interspike interval distributions for tape 7, run 4 (stimulation at 2-sec. inter- 
vals) and run 3 (no stimulus), unit 2L. (a) Stimulated run 7-4-2L: theoretical mean inter- 
val = 87.5 msec.; m = 2091 (including 92 shock artifacts); 6 per cent intervals exceed 200 
msec. (b) Spontaneous run 7-3-2L: theoretical mean interval = 88.5 msec.; m = 3072; 6 
per cent intervals exceed 200 msec. 


sion respectively of the two units—see average patterns of response revealed 
by traveling gate in FIGURE 7, runs 7-4-1 and 7-4-2L). 

Coincidences can be counted between spontaneous discharges of two neurons 
that are recorded by one or by two microelectrodes. The distance between 
the somata of two cells recorded with one electrode is likely to be small, but 
the exact dimension is unknown. Mountcastle ef al. (1957) suggest that an 


ee ee 


40 > 1.0 40 > 1.0 
a9 a2 
ae 
x Sens Recovery Curve ae Modified Recovery 
WW 30 25 30 £6 urve 
= 
z 50 10 20 
; 30 40 50 
4 20 20 msec 
< 
LS 
e 
e 10 {0 (b) 
6) 0 
0 40 80 120 160 200 O40 680) 120) 160) 200 
INTERSPIKE INTERVALS INTERSPIKE INTERVALS 


Ficure 2. Theoretical interspike interval distributions, using two assumed recovery 
curves. Histograms were calculated from a Poisson distribution of spikes modified by a 
multiplicative factor (labeled “Probability of Firing”) obtained from the indicated recovery 
curve as a function of the time since the last spike. 
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_ Interspike interval distributions for tape 7, run 4 (stimulation at 2-sec. in- 
en run 3 Go stimulus), unit 1. (qa) Stimulated run 7-4-1: theoretical mean in- 
terval = 440 msec.; n = 433 (excluding shock artifacts); 69.5 per cent intervals within 0-200 
msec. range; 6.5 per cent intervals within 200-300 msec. range; 4.5 per cent intervals within 
300-400 msec. range; 2.5 per cent intervals within 400-500 msec. range, 17.0 per cent inter- 
vals exceed 500 msec. (6) Spontaneous run 7-3-1: theoretical mean interval = 1.6 oe 
n = 173; 40 per cent intervals within 0-200 msec. range; 8 per cent intervals within a 
msec. range; 5 per cent intervals within 300-400 msec. range; 4 per cent intervals within 
400-500 msec. range; 43 per cent intervals exceed -500 msec. 
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initially negative unit has a minimum field of about 100 ». By contrast, the 
positive-negative spike is recorded only when the electrode tip is close to the 
membrane (Amassian, 1953; Rose and Mountcastle, 1954). When two posi- 
tive-negative spikes are recorded by a given microelectrode, the somata are 
presumably no more than a few microns apart. Two coincidence plots for 
spontaneous activity of two neurons that lie close to one another (run 7-3-2) 
or that are contiguous (run 7-2-1) are shown in FIGURE 4, In both pairs of 
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Ficure 4. Coincidences as a function of coinci i i 

f incidence interval for tape 7, run 3, units 2L 
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one electrode position: both (—) spikes; 7-3- i 
CA aay A ed ~) ore 8; run 2. (b) Two units recorded at one electrode 


neurons, the increase in number of coincidences is nearly linear for coincidence 
intervals up to 50 msec. The expected number of coincidences, calculated on 
the basis of a uniform random distribution of the spikes and an independent 
relationship between neurons, does not significantly differ from the observed 
number of coincidences. We can conclude from this that spontaneous dis- 
charges of the two pairs of neurons illustrated were independent; this state- 
ment, however, must be qualified. Above an interval of 10 msec. coitéideness 
were sought only by increasing the interval in steps of 10 ae Such time 
slices may average out an excitatory-inhibitory relationship. (For example 
when both the largest Group I fibers and smaller fibers of a muscle nerve aie 
stimulated, motoneurons supplying a synergist are monosynaptically facilitated 


Amassian et al.: Midbrain Reticular Formation 889 


and 0.5 msec. later are disynaptically inhibited [Lloyd, 1952].) For intervals 
less than 5 msec., the two pairs of units illustrated showed no greater coinci- 
dences than would be expected between independent discharges whether such 
coincidences were detected by the coincidence gate or from the oscilloscopic 
trace. The latter test revealed 32 coincidences within 0.8 msec. of the initial 
limb of the large spike (a). Such observations do not exclude the possibility of a 
delayed excitatory-inhibitory relationship. However, it is of interest that an 
excitatory-inhibitory sequence in the response of a given unit to peripheral 
stimulation can be demonstrated even when using a traveling 10-msec. gate. 
It seems unlikely in a complex polysynaptic network such as the reticular 
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FicureE 5. Coincidences as a function of coincidence interval for tape 7, ii, 

and 2, recorded with different microelectrodes. Run 7-11: O——O observed coincidences; 
@-—-—-@ expected random coincidences. 


formation that the transition from excitation to inhibition would occur as 
precisely as Lloyd (1952) showed for the motoneuron synapse. 

The coincidence plots for the units illustrated in FIGURE 5 are derived from 
midbrain reticular neurons that are about 1 to 119 mm. apart. The neurons 
appear to be independent in their spontaneous activity. 

We have sought long-term relationships between reticular neurons by com- 
paring average number of discharges of the two units during successive 10- or 
5-sec. periods. An example is shown in FIGURE 6. Between periods 35 to 53, 
the spike count in each 5-sec. period usually changed in the same direction. 
Tf indeed the two neurons had been studied only during this period, one might 
have inferred that they were functionally related. However, the units sOnae 
little relationship earlier in the run. The similarity in behavior during part 
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of the run might be due to the chance locking in of two rhythms with differing 
periodicities. Alternatively, a functional relationship between the two units 
may have been temporarily established. The need for making observations 
over an extended period of time before concluding that two neurons are func- 
tionally related is too obvious to require further emphasis. 


Effects of Stimulation 


Brief electrical stimuli were delivered to a paw at the rate of once every 2 
sec. through bipolar needles inserted into the skin. One effect of stimulation 
is to alter the distribution of interspike intervals (FIGURES 1a and 3a). A 
much more informative type of analysis is to count the number of discharges 
by a given neuron occurring during a gate of fixed duration that is opened by 
a pulse synchronized with each stimulus in the run. Spike counts are subse- 
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_FicureE 6. Variations in firing rate for two initially negative units recorded with one 
microelectrode; tape 7, run 3, units 2L and 2S. No stimulus (run 7-3-2). 


quently determined when the opening of the gate is delayed by successive in- 
crements equal to the fixed duration of the gate. Data obtained by this tech- 
nique from 3 simultaneously recorded neurons are shown in FIGURE 7. On 
the ordinate is plotted, for each gate delay, the value of the function 

O-E 

ng sesh where Es = tas. and 
O is the cumulative number of discharges by a neuron during the 10-msec. 
gate; Hs is the expected number of spontaneous discharges; Dg is the duration 
of each gate; V is the total number of spontaneous discharges during a stimu- 
lus free run; De is the duration of the spontaneous run; Ds is the duration of 
the stimulus run; P is the interval between stimuli. 

The delays for closure of the gates are shown on the abscissa. Throughout 
the run, a fixed 2-msec. delay was inserted between the stimulus artifact and 
the indicated times. This is insignificant as a source of error and is disre- 
garded except when very early responses are discussed. It should be noted 
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that Es is calculated on the basis that spontaneous discharges are uniformly 
distributed during a run in which no stimuli are given. This appears to con- 
tradict the statements made earlier concerning the distribution of intervals 
during a spontaneous run, but the value obtained from assuming a uniform 
distribution is of course the same as the expected value for a 10-msec. period 
from the observed distribution. The Es is a convenient abstraction that facili- 
tates comparison of the behavior of different neurons. It should be noted 
that the use of the expression (O — Es)/Es leads to an apparent lack of promi- 
nence of inhibitory effects, since the expression assumes a limiting value of —1 
when inhibition is complete. The effect of stimulating the ipsilateral and 
contralateral forepaws are analyzed in runs 7-4 and 7-6. The effect of the 
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Ficure 7. Firing patterns for three units when the site of stimulation is changed. Left 
column obtained with ipsilateral forepaw stimulation, right column with contralateral stimu- 
lation. Top and middle rows were obtained from two units recorded with the same micro- 
electrode, bottom row from a unit simultaneously recorded with another electrode. 


stimulus on the unit in runs 7-4-1 and 7-6-1 persists for at least 520 msec. 
after the stimulus. Furthermore, the behavior of the neuron is not uniform 
throughout the period of 520 msec. following the stimulus. This unit shows a 
phase of late response to stimulation of either paw, but the pattern of early 
activity differs according to the site of stimulation. The upper rows show 
data from two units recorded by the same microelectrode. The responses of 
both units show cyclical phases of facilitation and depression. The responses 
in runs 7-4-2L and 7-6-2L peak at about 60 msec. and again at 200 to 240 
msec. Peak response occurs earlier in the top row of plots. 

Responses of another pair of neurons to stimulation of the contralateral and 
ipsilateral forepaws, respectively, are analyzed in FIGURE 8. The predominant 
effect of stimulating the contralateral forepaw is either partial or complete 
inhibition of spontaneous discharge of both neurons for a period of at least 
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420 msec. after the stimulus. Moreover, the total number of discharges oc- 
curring during the 420-msec. period after the stimulus (O) is considerably less 
than the number (£) calculated on the basis that the discharges occurring 
during the stimulus run are uniformly distributed during the period between 
stimuli (compare O/E in runs 7-9-1 and 7-9-2). The behavior of the unit in 
run 7-10-2, following stimulation of the ipsilateral forepaw, shows a complex 
sequence of inhibition-excitation-inhibition for the period up to 420 msec. af- 
ter the stimulus. We are unable to determine if the other unit (top row) was 
excited or inhibited when the ipsilateral forepaw was stimulated (run 7-10-1). 
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_ Ficure 8. Firing patterns for two units recorded with two microelectrodes, when the 
site of stimulation is changed. Right column is obtained with ipsilateral stimulation, left 
column with contralateral stimulation. 


It is apparent that the above method of analysis does not discriminate be- 
tween repetitive responses of the neuron on a few occasions and single re- 
sponses on more numerous occasions, during a specified 10-msec. period after 
the stimulus. Nevertheless, reticular neurons often show repetitive discharge 
at high frequency (Amassian and Waller, 1958). Such repetitive discharges 
can be identified by counting only the first spike to occur in each gate period. 
This total is compared with the total obtained by counting all the spikes dur- 
ing the gate period; this check was performed for two units, and the difference 
in i ow clearly indicated the presence of such repetitive discharges by one 
unit. 

When responses are evoked from a pair of neurons within a given period 
after the stimulus, the number of coincidences occurring in this period is ob- 
viously greater than the number of random coincidences occurring in a period 
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of equal duration during a spontaneous run. While an increase (or decrease) 
in the number of coincidences in a specified period may be of great importance 
in the analysis of incoming signals by the nervous system, such observations 
do not necessarily indicate a functional relationship between the neurons. 
Coincidences during stimulus runs were counted for three different pairs of 
neurons, but only one of the pairs, recorded by a single microelectrode, yielded 
data from which conclusions can be drawn (see TABLE 1). Only a single dis- 
charge per gate was permitted to reach the counters. Discharges of neuron S 
were separately counted on one channel by electronically canceling the larger 
spike of the L neuron. The probability of discharge (Ps or Pz) of neuron S 
or L for a given gate delay was estimated by dividing the cumulative count 
during the gate by the number of stimuli (n = 93). The actual probability 
of coincidence was estimated by dividing the coincidence count by the number 


TABLE 1 
COINCIDENCE OF RESPONSES TO STIMULUS 
Run 7-4-2: Two Units Recorded at Same Position of Electrode 


Traveling 10-msec. Pg Pa Calculated Ps X Pz ence 
Start 10 msec. 0.27 Ovi 0.03 0.05 
20 0.78 0.09 0.07 0.08 
30 0.58 0.64 0.04 0.03 
40 Oe 22 O22 0.05 0.10 
50 O27 0.42 Oe tal 0.09 
60 0.42 0.34 0.14 Os13 
70 0.35 0.19 0.07 0.06 
80 0.35 0.12 0.04 0.05 
90 0.15 0.05 0.01 0.01 
S gate 20, L gate 0.77 0.46 0.35 0.37 
50 


of stimuli. If the discharge of the two neurons is independent, the product 
Ps: Py, gives the probability of random coincidence. Except in one instance, 
the actual probability of coincidence agrees well with the calculated proba- 
bility of random coincidence. The agreement also holds when the 10-msec. 
gates for the S and L units are started 20 and 50 msec., respectively, after the 
stimuli (these times were chosen because the peak responses of the two neu- 
rons occur prior to the closure of these gates). 


Discussion 


Our preliminary analysis of the effect of a peripheral stimulus on midbrain 
reticular neurons indicates that the spontaneous pattern of discharge is per- 
turbed for a considerable time after the stimulus. The changes may be subtle 
and may be missed by visual inspection of the spikes. The effect of a stimu- 
lus is nonuniform during the period between stimuli. Cyclical facilitation 
and depression of responses is revealed by using a computer for the analysis 
of the data. A spontaneously active neuron cannot be said to be either ex- 
cited or depressed following stimulation. Instead, the effect of stimulation 
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should be specified for a series of times after the stimulus. Two neurons re- 
corded simultaneously do not respond in an identical manner to a given stimu- 
lus. Peak responses may occur at different times after the stimulus regardless 
of whether the neurons are recorded by a single electrode, or whether they 
are recorded by two electrodes situated 1 to 2 mm. apart. 

The distribution of interspike intervals during spontaneous runs has already 
been discussed. The distribution is complex and cannot be fitted by a simple 
independent random model. Our conclusions agree with those derived by 
Kuffler ef al., (1957) from a study of spontaneous discharge of retinal ganglion 
cells. 

Failure to detect coincidences between spontaneous discharges of neurons 
that lie close to one another, other than those expected on a random basis, 
requires comment. Our technique of analysis may be too crude to reveal 
subtle relationships between neurons. Discharge of a reticular neuron pre- 
sumably requires bombardment by impulses in more than one presynaptic 
fiber. The number of active fibers is probably a small fraction of the total 
number of presynaptic fibers related to the neuron. The variability in be- 
havior of single reticular neurons implies that the set of active presynaptic 
fibers varies within the population of fibers impinging on a given reticular 
neuron. Such factors would be expected to reduce the likelihood of detecting 
a significant relationship between discharges of two neurons picked at random 
by a single microelectrode. It was previously inferred from the differences in 
behavior of neurons that lie close to one another that excitation of the reticu- 
lar net was not transmitted by sequential spread between adjacent neurons 
(Amassian and DeVito, 1954). If significant anatomical connections between 
reticular neurons occur at some distance from the somata (Scheibel and Schei- 
bel, 1958), the chances of recording related activity in two neurons is still 
further diminished. 


Summary 


The activities of simultaneously recorded midbrain reticular neurons are 
analyzed with a computer. A comparison is made between spontaneous and 
evoked activity in neurons that lie close to one another and between neurons 
that are separated by 1 to 2 mm. 
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